arXiv Machine Learning

Heterogeneous Agent Collaborative Reinforcement Learning

arXiv AI
Sep 28

G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies

G2MAF is a test‑time refinement framework for offline multi‑agent reinforcement learning that applies a single globally normalized, projected critic gradient to adjust all agents’ actions while keeping them close to a frozen policy proposal. The method improves performance on 24 Multi‑Party Environment (MPE) and StarCraft Multi‑Agent Challenge (SMAC) benchmarks, achieving mean relative gains of 9.2% on MPE and 8.9% on SMAC, with only a 6% increase in inference latency.

By Guowei Zou, Haitao Wang, Guoxin Wang, Zhiquan Chen, Beiwen Zhang, Guojie Wang, Hejun Wu
arXiv AI
Jun 9

Counterfactual Credit Policy Optimization for Multi-Agent Collaboration

arXiv:2603. 21563v4 Announce Type: replace Abstract: Collaborative multi-agent large language models (LLMs) can solve complex reasoning tasks by decomposing roles, but reinforcement learning for such systems is limited by credit assignment: shared terminal rewards obscure individual contributions and can encourage free-riding.

By Zhongyi Li, Wan Tian, Yikun Ban, Jinju Chen, Huiming Zhang, Yang Liu, Fuzhen Zhuang
arXiv Machine Learning
Oct 28

FlexMARL: Graph-Driven Rollout-Training Co-Design for LLM-Based Multi-Agent Reinforcement Learning

FlexMARL is a graph-driven rollout‑training co‑designed framework that addresses the infrastructure gap in large‑scale LLM‑based multi‑agent reinforcement learning. It introduces a multi‑agent execution graph (MEG) and a MEG Store for decentralized storage, a Rollout Engine that enables dependency‑parallel sampling with hierarchical load balancing, a Joint Orchestrator that uses an agent‑level asynchronous pipeline to reduce synchronization barriers, and a Training Engine that allocates resources on demand and swaps training states via location‑agnostic APIs. Experiments on a production cluster show up to 7.3× speedup and 5.6× better hardware utilization compared to existing frameworks.

By Zhida Jiang, Zhaolong Xing, Jiawei Lu, Yipei Niu, Qingyuan Sang, Liangxu Zhang, Wenquan Dai, Junhua Shu, Jiaxing Wang, Qiangyu Pei, Qiong Chen, Xinyu Liu, Fangming Liu, Ai Han, Zhen Chen, Ke Zhang
arXiv Computation and Language
Oct 19

MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation

arXiv:2610.23234v1 Announce Type: new Abstract: In a hierarchical multi-agent system, a Leader assigns tasks and subagents carry them out. Jointly training the Leader and the subagents is difficult b...

By Yuhan Sun, Jiaxin Zhao, Yongdong Luo, Yibo Hu, Meiguang Jin, Junfeng Ma, Weixun Wang, Weihang Pan, Zhongan Bi