arXiv Machine Learning

Objective-Behavior Alignment: Diagnostics for MORL Policy Selection

The paper introduces a diagnostic workflow for multi‑objective reinforcement learning (MORL) that reveals behavioral differences among policies on the Pareto front, which are not apparent from value vectors alone. It offers quantitative and visual tools to inspect these variations and demonstrates their effectiveness on both simple grid tasks and more complex continuous‑control benchmarks.

arXiv Machine Learning
Oct 28

Preference-Guided Cooperation in Multi-Objective Multi-Agent Reinforcement Learning

The paper introduces Meta-Gradient Informed Multi-Objective Multi-Agent Reinforcement Learning (MIRL), a framework that leverages the structure of multiple objectives to enhance cooperation among agents. By formulating MOMARL at both team and individual levels and using objective-wise counterfactual credit assignment, MIRL allows agents to learn how to prioritize objectives individually while jointly optimizing a system-level objective. Experiments on a multi-objective particle environment demonstrate that MIRL, especially its dynamic variant, improves Pareto-front quality compared to baseline methods.

By Qiyue Xia, J. Michael Herrmann
arXiv AI
Oct 19

Coordination Matters: Process-Aware Evaluation of Cooperative Multi-Agent Reinforcement Learning

The paper introduces a process‑aware evaluation framework for cooperative multi‑agent reinforcement learning that supplements traditional outcome metrics with diagnostics of interference, coordinated task completion, participation, and coordination efficiency. Using the STAT testbed and two established benchmarks, the authors demonstrate that policies with similar returns can differ markedly in coordination behavior, and that reward shaping can alter coordination without affecting overall task return. The framework reveals distinct process signatures across spatial, task, and agent scaling and remains informative across different coordination mechanisms.

By Maria Ana Cardei, Matthew Landers, Afsaneh Doryab
arXiv AI
Oct 27

Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution

The paper introduces objective‑conflict coefficients to measure how competing goals affect learning efficiency in multi‑objective reinforcement learning. It proposes a routed ensemble policy algorithm that keeps a small set of basis policies and routes trajectories to the most promising one, and a kernel surrogate trajectory attribution method that estimates how feedback from one trajectory influences learning on others. Experiments on Meta‑World and MO‑Gymnasium show routed ensembles boost average performance by 4.3% and the kernel attribution improves correlation with leave‑one‑out analysis by 50%, achieving a 27% reward gain over baseline attribution methods.

By Zhenshuo Zhang, Ruoxuan Xiong, Hongyang R. Zhang