arXiv AI

MASTARS: Multi-Agent Sequential Trajectory Augmentation with Return-Conditioned Subgoals

arXiv AI
Oct 7

One for All, All for One: Coordinated Multi-Agent Diffusion Steering via Stochastic Optimal Control

The paper introduces Coordinated Multi-Agent Diffusion Steering (CMDS), a framework that reuses independently trained diffusion models as generative primitives and learns a control policy to coordinate their reverse processes. By framing coordination as a stochastic optimal control problem, CMDS balances an assembly-level reward with adherence to pretrained dynamics, enabling the learned control to generalise across new tasks. Experiments demonstrate CMDS’s ability to recover target distributions, satisfy spatial constraints, and separate sources from degraded mixtures in applications such as maze navigation, robot planning, and text-conditioned human motion.

By Riccardo Barbano, Vincent Pauline, Runchang Li, George Webber, Alexander Denker, \v{Z}eljko Kereta, Stefan Bauer, Francisco Vargas, Esmeralda S. Whitammer
arXiv AI
Sep 4

Out-of-Distribution Generalisation with Sequence Models in Offline Multi-Agent Reinforcement Learning

The paper investigates zero‑shot task generalisation in offline multi‑agent reinforcement learning by extending sequence‑modeling architectures to support multi‑task observation and action spaces and variable agent counts. It finds that increasing task diversity, rather than merely enlarging the dataset, is the key driver for robust zero‑shot transfer. Experiments on four challenging environments show a 3.2× mean improvement on held‑out tasks compared to single‑task models and outperform strong behaviour‑cloning baselines.

By Oussama Hidaoui, Omer Ebead, Ulrich Armel Mbou Sob, Siddarth Singh, Juan Claude Formanek, Felix Chalumeau, Omayma Mahjoub, Sasha Abramowitz, Ruan John de Kock, Wiem Khlifi, Louay Ben Nessir, Simon Verster Du Toit, Daniel Rajaonarivonivelomanantsoa, Asim Awad Osman, Arnol Manuel Fokam, Refiloe Shabe, Arnu Pretorius
arXiv Machine Learning
Oct 5

Test-time Multi-agent Coordination by Decomposed Value Gradient Flow

The paper introduces SCOUT, an offline multi-agent reinforcement learning framework that combines a generative behavioral prior with a decomposed value function for test-time action refinement. SCOUT uses optimal unified transport and Stein variational gradient descent to steer behavioral samples toward high-value regions, with the number of transport steps providing adaptive scaling. The authors prove a single-term KL bound on the joint soft-value gap under the individual-global-max principle and demonstrate that SCOUT outperforms existing methods on both discrete and continuous offline MARL benchmarks, including offline-to-online settings.

By Dongsu Lee, Haoran Xu, Amy Zhang
arXiv AI
Sep 18

Dual-Axis Policy Optimization for LLM Agents: Bayesian Feedback Attribution and Trajectory Mass Normalization

The paper introduces BATON, a dual‑axis policy optimization framework for large language model agents that addresses two key aspects of reinforcement learning: intra‑trajectory feedback attribution and inter‑trajectory objective aggregation. BATON combines Bayesian Feedback Attribution, which builds a feedback‑conditioned posterior over sampled actions, with Trajectory Mass Normalization, which ensures equal optimization weight for complete trajectories. Experiments on ALFWorld, WebShop, and SearchQA demonstrate that each axis independently improves performance and that their combination yields the best results across different model scales.

By Yingxuan Zhuang, Binhe Yu, Jingxiao Yang, Ruopei Sun, Ziting Li, Cheng Tan, Xuhong Zhang, Jianwei Yin, Jintao Chen
arXiv AI
Oct 2

Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policies

The paper introduces OMAF, an Online MARL framework that uses a one-step Transformer-based flow policy to generate coordinated actions efficiently. By replacing costly iterative sampling with a single-step action generation and a joint optimization scheme that couples softmax Q-value estimation with flow policy objectives, OMAF maintains expressive multimodal behavior while improving training speed. Experiments on 10 tasks from MPE and MAMuJoCo demonstrate up to 3.4× higher returns and 10.5× better sample efficiency compared to baseline methods.

By Zhuoran Li, Yunzhan Li, Xun Wang, Yihan Du, Longbo Huang