The paper introduces a process‑aware evaluation framework for cooperative multi‑agent reinforcement learning that supplements traditional outcome metrics with diagnostics of interference, coordinated task completion, participation, and coordination efficiency. Using the STAT testbed and two established benchmarks, the authors demonstrate that policies with similar returns can differ markedly in coordination behavior, and that reward shaping can alter coordination without affecting overall task return. The framework reveals distinct process signatures across spatial, task, and agent scaling and remains informative across different coordination mechanisms.
arXiv:2602. 20804v2 Announce Type: replace Abstract: Cooperative multi-agent reinforcement learning (MARL) is typically framed as a decentralised partially observable Markov decision process (Dec-POMDP), a setting whose hardness stems from two key challenges: partial observability and decentralised coordination.
arXiv:2610.18808v1 Announce Type: cross Abstract: In cooperative Multi-Agent Reinforcement Learning (MARL), agents must learn to coordinate their actions to achieve shared objectives. A key indicator...
arXiv:2502. 00345v2 Announce Type: replace-cross Abstract: The critical role of division of labor (DOL) in enhancing cooperation is well-recognized in real-world applications.
The paper introduces a diagnostic workflow for multi‑objective reinforcement learning (MORL) that reveals behavioral differences among policies on the Pareto front, which are not apparent from value vectors alone. It offers quantitative and visual tools to inspect these variations and demonstrates their effectiveness on both simple grid tasks and more complex continuous‑control benchmarks.
arXiv:2609.06586v1 Announce Type: cross Abstract: A shared reward gives agents a common objective, but leaves open when, how and even whether they must cooperate to succeed. We address these question...
The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.
arXiv:2511. 02304v2 Announce Type: replace-cross Abstract: We study learning multi-task, multi-agent policies for cooperative, temporal objectives, under centralized training, decentralized execution.
arXiv:2602. 05459v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method.
The paper introduces self‑referenced social preferences, allowing agents to learn cooperative behavior without observing others’ private rewards. Each agent models its own reward, applies this model to observed transitions of other agents, and uses the resulting assessments to inform either the learning reward or policy updates. Experiments on three social‑dilemma environments show that agents can achieve cooperation and more equitable outcomes even when independent learners fail, with the best integration point depending on the type of social preference used.
arXiv:2606. 08340v1 Announce Type: new Abstract: As language models are increasingly deployed as autonomous agents, they must coordinate with others over long horizons in open-ended interactive tasks.
The paper introduces SCOUT, an offline multi-agent reinforcement learning framework that combines a generative behavioral prior with a decomposed value function for test-time action refinement. SCOUT uses optimal unified transport and Stein variational gradient descent to steer behavioral samples toward high-value regions, with the number of transport steps providing adaptive scaling. The authors prove a single-term KL bound on the joint soft-value gap under the individual-global-max principle and demonstrate that SCOUT outperforms existing methods on both discrete and continuous offline MARL benchmarks, including offline-to-online settings.
The paper introduces Turnover-Orthogonal Credit Assignment (TOCA), a method for open-team multi-agent reinforcement learning that separates the effects of agent actions, pure turnover, and action–turnover interactions in the team return. TOCA uses a permutation‑invariant centralized critic with event tokens to provide counterfactual per‑agent credit signals and a softly weighted interaction variant, TOCA‑β, for high‑variance environments. Experiments show that TOCA outperforms event‑aware MAPPO critics and that removing interaction credit degrades performance, especially in high‑turnover settings such as the Dynamic Spread benchmark.