The paper investigates learning Nash equilibria in partially observable Markov games (POMGs) where agents cannot fully observe the state. By focusing on a subclass with independent state transitions and a Markov potential game structure, the authors propose an independent learning algorithm that allows agents to converge to an approximate Nash equilibrium using only their own observations and actions, without communication. Under a filter stability assumption, finite‑history policies are shown to approximate the POMG sufficiently, enabling a surrogate near‑potential Markov game and yielding quasi‑polynomial sample and computational complexity.
By Philip Jordan, Maryam Kamgarpour
arXiv:2607. 08012v1 Announce Type: cross Abstract: This paper studies an online variant of the assistance games framework, where an informed agent and an uninformed agent repeatedly interact over $T$ timesteps to optimize a common reward function.
By Nivasini Ananthakrishnan, Mark Bedaywi, Michael I. Jordan, Stuart Russell, Nika Haghtalab
The paper presents an algorithm that lets a learning agent ask for help from a mentor and transfer knowledge between similar states, enabling safe and effective learning in Markov decision processes with irreversible dynamics and infinite state spaces. It proves that both regret and the number of mentor queries grow sublinearly over time, using a sequence of three reductions to achieve a general result. The work claims to be the first formal proof that an agent can achieve high reward while becoming self‑sufficient in an unknown, unbounded, high‑stakes environment without resets.
By Benjamin Plaut, Juan Li\'evano-Karim, Hanlin Zhu, Stuart Russell
arXiv:2409. 01447v3 Announce Type: replace Abstract: We present a finite-sample analysis of decentralized learning in two-player zero-sum matrix games and stochastic games, with a focus on best-response-based learning algorithms.
By Zaiwei Chen, Kaiqing Zhang, Eric Mazumdar, Asuman Ozdaglar, Adam Wierman
We introduce the first Probably Approximately Correct (PAC) learning framework for general-sum concurrent stochastic games (CSGs) with transition uncertainty, while addressing the challenge of Nash equilibrium (NE) existence. Our algorithm maintains data-driven $L^1$ confidence sets over transition kernels and solves a robust CSG to compute a social-welfare optimal $\varepsilon$-NE, using a robust MDP-based exploration mechanism to drive joint state-action coverage.
arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.
By Jongmin Lee, Ernest K. Ryu, Vaneet Aggarwal
arXiv:2608. 09389v1 Announce Type: cross Abstract: This note aims to serve as an entry point to the literature on learning in games, a topic with significant theoretical appeal and a wide range of applications -- from machine learning and data science to economics and beyond.
By Panayotis Mertikopoulos
The paper presents the first PAC learning framework for general-sum concurrent stochastic games with uncertain transitions, addressing the challenge of Nash equilibrium existence. It introduces data‑driven L¹ confidence sets over transition kernels and a robust CSG solver that computes a social‑welfare optimal ε‑NE, or provides a certificate that no exact NE exists. The algorithm achieves polynomial sample complexity under a minimum reachability condition and is validated on benchmark CSGs with near‑optimal performance.
By Angel Y. He, David Parker
arXiv:2606. 11284v1 Announce Type: cross Abstract: Real-world multi-agent systems, from traffic coordination to resource allocation, are often modeled as general-sum games where individual incentives conflict with collective welfare.
By Wongyu Lee, Francesco Lelli, Omran Ayoub, Massimo Tornatore
The paper introduces aspiration-based perturbed learning automata (APLA), a payoff‑based learning scheme that incorporates an aspiration factor to reinforce action selection in distributed multi‑player games. It presents a stochastic stability analysis of APLA in positive‑utility games with noisy observations, establishing that the infinite‑dimensional Markov chain induced by the dynamics can be reduced to a finite‑dimensional one. This work extends previous results beyond potential and coordination games to generic non‑zero‑sum games, with a second part focusing on weakly acyclic games.
By Georgios C. Chasparis
arXiv:2609. 14959v1 Announce Type: new Abstract: We study decentralized learning of Nash equilibria (NE) in infinite-horizon discounted Markov games under bandit feedback, focusing on Markov $\alpha$-potential games.
By S. Rasoul Etesami
arXiv:2608. 10529v1 Announce Type: cross Abstract: The multi-armed bandit problem is a central framework in sequential decision-making, extensively studied under sub-Gaussian reward assumptions.
By Daphne Feng, Ricardo Parada, Lily Jiang, Sophia Yi, William Chang