The paper introduces EFXPLORER, a constrained exploration framework for cooperative multi-agent reinforcement learning that maximizes exploration objectives while preserving task performance. It uses an epigraph reformulation to create adaptive exploration budgets, separating intrinsic rewards from task objectives and regulating exploration through task feasibility. Additionally, a successor distance-based intrinsic reward is incorporated to promote diverse, temporally extended exploration, leading to improved performance and novel cooperative strategies on SMAX, VMAS, and MPE benchmarks.
By Sungil Son, Hoseong Jung, Dahyun Oh, H. Jin Kim
arXiv:2608. 10499v1 Announce Type: cross Abstract: Personalized Federated Reinforcement Learning (PFRL) takes a decentralized approach to storing and accessing information based on past experiences while keeping each client's data private during the learning of each client's policy.
By Md Rafid Islam, Rafsan Jany, Zahid Hasan, Ratun Rahman
Personalized Federated Reinforcement Learning (PFRL) takes a decentralized approach to storing and accessing information based on past experiences while keeping each client's data private during the learning of each client's policy. Many current methods for PFRL rely heavily on exploiting existing reinforcement learning reward signals to derive an optimal policy for each client, thereby neglecting exploration in non-stationary or sparse-reward environments.
arXiv:2606. 03962v1 Announce Type: cross Abstract: Classical reinforcement learning (RL) typically seeks a deterministic policy that maximizes the expected sum of a scalar reward.
By Anthony GX-Chen, Ankit Anand, Gheorghe Comanici, Zaheer Abbas, Eser Ayg\"un, David Smalling, Shibl Mourad, Doina Precup, Andr\'e Barreto, Mark Rowland
Classical reinforcement learning (RL) typically seeks a deterministic policy that maximizes the expected sum of a scalar reward. Yet, modern applications such as language model fine-tuning or scientific discovery demand diversity.
arXiv:2505. 08630v2 Announce Type: replace Abstract: Training cooperative agents in sparse-reward scenarios poses significant challenges for multi-agent reinforcement learning (MARL).
By Shuai Han, Mehdi Dastani, Shihan Wang
arXiv:2607. 08647v1 Announce Type: cross Abstract: As autonomous agents are increasingly deployed across diverse operational contexts, aligning their behavior with human intent demands reward functions that remain robust to such changes rather than overfitting to any single environment.
By Ali Larian, Qian Lin, Chang Zong Wu, Daniel S. Brown
The paper investigates when intrinsic rewards effectively drive exploration in reinforcement learning. It introduces a formal criterion that evaluates policies based on the counterfactual information they acquire, comparing how well their histories can replace experience from alternative policies. Using a simple environment, the authors show that common intrinsic reward objectives—count-based, prediction-error, empowerment, and information-gain—can lead to Pareto-suboptimal exploration under this criterion, and they propose conditions and a new objective that better align with optimal exploration.
By Scott W. Viteri (Stanford University), Laura Gomezjurado Gonzalez (Stanford University), Clark Barrett (Stanford University)
arXiv:2509. 10656v2 Announce Type: replace-cross Abstract: For groups of autonomous agents to achieve a particular goal, they must engage in coordination and long-horizon reasoning.
By Chirayu Nimonkar, Shlok Shah, Catherine Ji, Benjamin Eysenbach
arXiv:2608. 08604v1 Announce Type: new Abstract: Multi-agent reinforcement learning (MARL) is a powerful framework for solving complex collaborative tasks, but it relies heavily on well-defined global reward functions.
By Ni Mu, Yao Luan, Yiqin Yang, Qing-Shan Jia
The paper introduces ELEMENT, a framework that combines episodic and lifelong entropy maximization to drive reward-free exploration in reinforcement learning. It addresses two key limitations of existing entropy-based methods: the vanishing intrinsic reward after a state is visited and the computational cost of estimating entropy over large datasets. ELEMENT achieves this by deriving an average episodic state entropy reward and employing a k‑NN graph‑based estimator for lifelong entropy, leading to superior state coverage and unsupervised pre‑training performance compared to current baselines.
By Hongming Li, Zhao Yang, Xiaoxuan Liang, Shujian Yu, Jose C. Principe
arXiv:2607. 19232v1 Announce Type: new Abstract: Hierarchical Reinforcement Learning (HRL) intends to separate strategic planning from primitive execution.
By Kshitij Kumar Srivastava, Kshitij Jerath