arXiv Machine Learning

Phi-Actor-Critic: Steering General-Sum Games to Pareto-Efficient Correlated Equilibria

arXiv:2606. 11284v1 Announce Type: cross Abstract: Real-world multi-agent systems, from traffic coordination to resource allocation, are often modeled as general-sum games where individual incentives conflict with collective welfare.

arXiv Machine Learning
Jun 5

DNQ: Deep Nash Q-Network for Partially Observable n-Player Games

arXiv:2606. 06480v1 Announce Type: cross Abstract: Many real-world competitive systems require multiple decision-makers to act simultaneously under shared constraints, limited information, and repeated interaction, as in auctions, resource allocation, and security competition.

By Qintong Xie, Edward Koh, Xavier Cadet, Peter Chin
arXiv Machine Learning
Sep 4

Towards Scaling Reinforcement Learning to Massive Populations: Learning Mean-Field Representations

The paper proposes a mean‑field reinforcement learning framework that models rewards and transitions as functions of an unknown low‑dimensional aggregate statistic of a large agent population. By learning this low‑dimensional representation in an offline setting, the authors demonstrate a provable method for obtaining near‑optimal policies. Experiments on a one‑step routing game inspired by supply‑chain problems show that, with a fixed neural‑network size and optimization budget, the learned representation improves reward prediction and the quality of Nash equilibria compared to baselines that ignore population structure.

By Aditya Makkar, Benjamin Unger, Jeongyeol Kwon, Mathieu Lauri\`ere, Eugene Vinitsky, Yonathan Efroni
arXiv Machine Learning
Aug 28

Shared Actors Need Not Share Critics: Effects of Value Mismatch in Parallel Reinforcement Learning

The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.

By Zhenya Liu, Yang Meng, Zhuokai Zhao, Xuefeng Liu, Yuxin Chen
arXiv Machine Learning
Sep 24

Optimization without Future Compromises? Decentralized Coordination via Collective and Reinforcement Learning

The paper introduces Hierarchical Reinforcement and Collective Learning (HRCL), a framework that combines multi‑agent reinforcement learning (MARL) with decentralized coordination. HRCL uses MARL at a high level to generate strategic guidance that limits the decision space for low‑level agents, enabling efficient short‑term coordination while considering long‑term effects. Experiments on synthetic, energy‑management, and drone‑swarm scenarios demonstrate faster convergence and significant reductions in system‑wide and individual costs compared to standalone MARL.

By Chuhao Qin, Evangelos Pournaras
arXiv AI
Jul 14

Beyond Bayesian Nash: Learning Minimax-Regret Equilibria for Adversarial Team Games under Asymmetric Information

arXiv:2607. 09993v1 Announce Type: cross Abstract: Adversarial team games (ATGs) with asymmetric information, such as adversarial path-finding, goal search, and reachability games on graphs, require strategies that are robust to hidden opponent types, such as a hidden goal flag, and to deception.

By Naman Aggarwal, Jonathan P. How