The paper investigates learning Nash equilibria in partially observable Markov games (POMGs) where agents cannot fully observe the state. By focusing on a subclass with independent state transitions and a Markov potential game structure, the authors propose an independent learning algorithm that allows agents to converge to an approximate Nash equilibrium using only their own observations and actions, without communication. Under a filter stability assumption, finite‑history policies are shown to approximate the POMG sufficiently, enabling a surrogate near‑potential Markov game and yielding quasi‑polynomial sample and computational complexity.
By Philip Jordan, Maryam Kamgarpour
arXiv:2607. 21876v1 Announce Type: new Abstract: We investigate a decentralized reinforcement learning problem involving multiple agents that interact with the same Markov Decision Process (MDP).
By Sreejeet Maity, Feng Zhu, Aritra Mitra, Robert W. Heath Jr
arXiv:2603. 14867v4 Announce Type: replace-cross Abstract: Many strategic decision-making problems, such as environment design for warehouse robots, can be naturally formulated as bi-level reinforcement learning (RL), where a leader agent optimizes its objective while a follower solves a Markov decision process (MDP) conditioned on the leader's decisions.
By Mikoto Kudo, Takumi Tanabe, Akifumi Wachi, Youhei Akimoto
The paper introduces Hierarchical Reinforcement and Collective Learning (HRCL), a framework that combines multi‑agent reinforcement learning (MARL) with decentralized coordination. HRCL uses MARL at a high level to generate strategic guidance that limits the decision space for low‑level agents, enabling efficient short‑term coordination while considering long‑term effects. Experiments on synthetic, energy‑management, and drone‑swarm scenarios demonstrate faster convergence and significant reductions in system‑wide and individual costs compared to standalone MARL.
By Chuhao Qin, Evangelos Pournaras
arXiv:2607. 19809v1 Announce Type: cross Abstract: In multi-agent reinforcement learning (MARL), inter-agent communication is effective for improving performance under partial observability.
By Taisuke Takayama, Naoto Yoshida, Tadahiro Taniguchi
The paper introduces Fed‑LSVI, a federated online reinforcement learning algorithm that uses linear function approximation in episodic Markov decision processes. It achieves a regret bound of ≥O(√{Md^3H^4T}) while only exchanging compressed sufficient statistics, thereby meeting privacy constraints. The method reduces communication cost to logarithmic in the number of episodes, a marked improvement over previous approaches that required linear communication.
By Zihang Liang, Haochen Zhang, Lingzhou Xue
arXiv:2609. 14959v1 Announce Type: new Abstract: We study decentralized learning of Nash equilibria (NE) in infinite-horizon discounted Markov games under bandit feedback, focusing on Markov $\alpha$-potential games.
By S. Rasoul Etesami
arXiv:2409. 01447v3 Announce Type: replace Abstract: We present a finite-sample analysis of decentralized learning in two-player zero-sum matrix games and stochastic games, with a focus on best-response-based learning algorithms.
By Zaiwei Chen, Kaiqing Zhang, Eric Mazumdar, Asuman Ozdaglar, Adam Wierman
The paper introduces a decentralized learning framework for finding socially optimal equilibria in finite normal-form games played over dynamic communication networks. Agents only observe their own payoffs, lack prior knowledge of the game, and communicate with time-varying neighbors using low-bandwidth, time-stamped tables instead of raw actions or payoff data. The proposed dynamics combine randomized semantic signals, table fusion, and temporal majority reconstruction to achieve finite-time logarithmic regret guarantees for optimal equilibrium selection under utilitarian and proportional-fair social welfare objectives, as demonstrated by simulations.
By Seref Taha Kiremitci, Muhammed O. Sayin
arXiv:2606. 30072v1 Announce Type: new Abstract: Cooperative tasks in Multi-Agent Reinforcement Learning (MARL) require agents to collectively maximize a shared return.
By Daiki E. Matsunaga, Junho Na, Tri Wahyu Guntara, Scott Sanner, Pascal Poupart, Jongmin Lee, Kee-Eung Kim
arXiv:2607. 14877v1 Announce Type: new Abstract: Reachability is the most fundamental logical objective, yet it is notoriously difficult to learn in reinforcement learning settings: even for Markov decision processes, PAC learning of reachability is impossible without additional assumptions.
By Ali Asadi, Krishnendu Chatterjee, Pavol Kebis
arXiv:2606. 12281v1 Announce Type: cross Abstract: In Decentralized Training and Decentralized Execution (DTDE) for cooperative Multi-Agent Reinforcement Learning (MARL), action-advising-based knowledge sharing promotes interpretable and scalable cooperation among agents.
By Jinyuan Zu, Xiaowei Lv, Yongcai Wang, Deying Li, Yunjun Han, Wenping Chen, Fengyi Zhang, Naiqi Wu