The paper investigates whether evolutionary stability guarantees that learning agents can achieve cooperative outcomes in a multi‑agent setting. Using a three‑agent governance game, the authors compare the evolutionary basin of attraction with learning basins derived from independent Q‑learning, scaled Boltzmann exploration, and SA–EA BQL. They find that while the evolutionary basin covers the entire sampled grid, only ε‑greedy Q‑learning attains a substantial learning basin, whereas the other methods fail to sustain cooperation, highlighting a disconnect between population‑level stability and finite‑sample learning accessibility.
By Yijie Wang
The paper presents a mean‑field framework for studying multi‑agent Q‑learning in networked populations, where agents update stateless Q‑values on a fixed graph while the average behavior of the population feeds back to modify the payoff matrix. A deterministic transport equation for the distribution of Q‑values is derived and coupled with a discrete update for the environmental state, and the model is validated against Monte Carlo simulations on several random graph topologies. Results show that the mean‑field system captures macroscopic cooperation dynamics, that environmental feedback reshapes action‑value ordering, and that the timescale of environmental response critically influences learning outcomes.
By Lichen Wang, Shijia Hua, Linjie Liu
arXiv:2601. 17454v2 Announce Type: replace-cross Abstract: Centralized value learning underlies a broad class of multi-agent reinforcement learning methods, but its claimed advantage is typically evaluated in settings that confound coordination structure with function approximation and partial observability.
By Muhammad Ahmed Atif, Nehal Naeem Haji, Mohammad Shahid Shaikh, Muhammad Ebad Atif
arXiv:2606. 11284v1 Announce Type: cross Abstract: Real-world multi-agent systems, from traffic coordination to resource allocation, are often modeled as general-sum games where individual incentives conflict with collective welfare.
By Wongyu Lee, Francesco Lelli, Omran Ayoub, Massimo Tornatore
The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.
By Zhenya Liu, Yang Meng, Zhuokai Zhao, Xuefeng Liu, Yuxin Chen
The paper introduces CURB, a reward‑shaping framework that penalizes the total variation distance between an agent’s action distributions under cooperation and defection histories, thereby preventing collusive equilibria in repeated games. By linking empirical Q‑learning collusion to Simple Penal Codes, the authors prove that any non‑trivial SPC can be neutralized, and demonstrate CURB’s effectiveness in both tabular and deep Q‑learning settings for Bertrand and Cournot competition.
By Karthik Sivachandran, Rohan Paleja
arXiv:2608. 13810v1 Announce Type: cross Abstract: We examine the interplay between ordinal, preference-based solution concepts in games and the long-run behavior of game dynamics, asking in particular to what extent the combinatorial data of a game -- its preference graph -- determine the outcomes of no-regret learning dynamics -- such as follow-the-regularized-leader (FTRL).
By Omar Abbadi, Rida Laraki, Panayotis Mertikopoulos
arXiv:2510. 14907v2 Announce Type: replace-cross Abstract: We extend the study of learning in games to dynamics that exhibit non-asymptotic stability.
By Geelon So, Yi-An Ma
The paper proposes a mean‑field reinforcement learning framework that models rewards and transitions as functions of an unknown low‑dimensional aggregate statistic of a large agent population. By learning this low‑dimensional representation in an offline setting, the authors demonstrate a provable method for obtaining near‑optimal policies. Experiments on a one‑step routing game inspired by supply‑chain problems show that, with a fixed neural‑network size and optimization budget, the learned representation improves reward prediction and the quality of Nash equilibria compared to baselines that ignore population structure.
By Aditya Makkar, Benjamin Unger, Jeongyeol Kwon, Mathieu Lauri\`ere, Eugene Vinitsky, Yonathan Efroni
arXiv:2606. 28824v1 Announce Type: cross Abstract: This paper develops a continuum theory of exit-and-join coalition dynamics in nonatomic cooperative games.
By Quanyan Zhu
arXiv:2607. 26788v1 Announce Type: cross Abstract: Clustered federated learning benefits from organizing heterogeneous participants into coalitions that train coalition-specific models, but such clustering is sustainable only if participants prefer their assigned coalition and the required transfers are affordable.
By Cengis Hasan
arXiv:2608. 09389v1 Announce Type: cross Abstract: This note aims to serve as an entry point to the literature on learning in games, a topic with significant theoretical appeal and a wide range of applications -- from machine learning and data science to economics and beyond.
By Panayotis Mertikopoulos