The paper investigates whether evolutionary stability guarantees that learning agents can achieve cooperative outcomes in a multi‑agent setting. Using a three‑agent governance game, the authors compare the evolutionary basin of attraction with learning basins derived from independent Q‑learning, scaled Boltzmann exploration, and SA–EA BQL. They find that while the evolutionary basin covers the entire sampled grid, only ε‑greedy Q‑learning attains a substantial learning basin, whereas the other methods fail to sustain cooperation, highlighting a disconnect between population‑level stability and finite‑sample learning accessibility.
Cooperation emergence is a central problem in multi-agent systems because decentralized agents must coordinate while adapting to the changing behavior of others. Evolutionary game theory identifies st...
The paper presents a mean‑field framework for studying multi‑agent Q‑learning in networked populations, where agents update stateless Q‑values on a fixed graph while the average behavior of the population feeds back to modify the payoff matrix. A deterministic transport equation for the distribution of Q‑values is derived and coupled with a discrete update for the environmental state, and the model is validated against Monte Carlo simulations on several random graph topologies. Results show that the mean‑field system captures macroscopic cooperation dynamics, that environmental feedback reshapes action‑value ordering, and that the timescale of environmental response critically influences learning outcomes.
arXiv:2601. 17454v2 Announce Type: replace-cross Abstract: Centralized value learning underlies a broad class of multi-agent reinforcement learning methods, but its claimed advantage is typically evaluated in settings that confound coordination structure with function approximation and partial observability.
The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.
arXiv:2606. 11284v1 Announce Type: cross Abstract: Real-world multi-agent systems, from traffic coordination to resource allocation, are often modeled as general-sum games where individual incentives conflict with collective welfare.
arXiv:2607. 26788v1 Announce Type: cross Abstract: Clustered federated learning benefits from organizing heterogeneous participants into coalitions that train coalition-specific models, but such clustering is sustainable only if participants prefer their assigned coalition and the required transfers are affordable.
The paper proposes a mean‑field reinforcement learning framework that models rewards and transitions as functions of an unknown low‑dimensional aggregate statistic of a large agent population. By learning this low‑dimensional representation in an offline setting, the authors demonstrate a provable method for obtaining near‑optimal policies. Experiments on a one‑step routing game inspired by supply‑chain problems show that, with a fixed neural‑network size and optimization budget, the learned representation improves reward prediction and the quality of Nash equilibria compared to baselines that ignore population structure.
The paper introduces CURB, a reward‑shaping framework that penalizes the total variation distance between an agent’s action distributions under cooperation and defection histories, thereby preventing collusive equilibria in repeated games. By linking empirical Q‑learning collusion to Simple Penal Codes, the authors prove that any non‑trivial SPC can be neutralized, and demonstrate CURB’s effectiveness in both tabular and deep Q‑learning settings for Bertrand and Cournot competition.
arXiv:2608. 13810v1 Announce Type: cross Abstract: We examine the interplay between ordinal, preference-based solution concepts in games and the long-run behavior of game dynamics, asking in particular to what extent the combinatorial data of a game -- its preference graph -- determine the outcomes of no-regret learning dynamics -- such as follow-the-regularized-leader (FTRL).
arXiv:2608. 12753v1 Announce Type: new Abstract: We study decentralized multi-player reinforcement learning in episodic tabular Markov decision processes (MDPs) under three forms of information asymmetry: (A) unobserved actions with common rewards, (B) observed actions with independent rewards, and (C) unobserved actions with independent rewards.
arXiv:2609.06467v1 Announce Type: new Abstract: In performative reinforcement learning the deployed policy shapes the environment that generates the learner's future data, and the natural solution co...
arXiv:2607. 04470v1 Announce Type: cross Abstract: Large Language Models (LLMs) offer a natural interface for translating human objectives into reward signals for cooperative multi-agent reinforcement learning (MARL), yet the training-time dynamics of this integration remain poorly understood.