The paper introduces ELEMENT, a framework that combines episodic and lifelong entropy maximization to drive reward-free exploration in reinforcement learning. It addresses two key limitations of existing entropy-based methods: the vanishing intrinsic reward after a state is visited and the computational cost of estimating entropy over large datasets. ELEMENT achieves this by deriving an average episodic state entropy reward and employing a k‑NN graph‑based estimator for lifelong entropy, leading to superior state coverage and unsupervised pre‑training performance compared to current baselines.
arXiv:2603.18965v2 Announce Type: replace Abstract: Maximum entropy reinforcement learning motivates agents to explore states and actions to maximize the entropy of some distribution, typically by pr...
arXiv:2608. 05111v1 Announce Type: new Abstract: In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies.
arXiv:2607. 29419v1 Announce Type: cross Abstract: In reinforcement learning, exploration with sparse and delayed rewards presents a significant challenge due to the limited feedback available for guiding the learning process.
arXiv:2606. 19476v1 Announce Type: cross Abstract: Effective machine learning depends not only on how we model data, but also on what data we choose to collect.
arXiv:2608. 10499v1 Announce Type: cross Abstract: Personalized Federated Reinforcement Learning (PFRL) takes a decentralized approach to storing and accessing information based on past experiences while keeping each client's data private during the learning of each client's policy.
The paper introduces two algorithms for personalized federated reinforcement learning: PF-UCBVI, which achieves optimal linear speedup by separating shared dynamics from individual goals, and PF-EFVI, a greedy method that uses reward diversity to guarantee state-action coverage without explicit exploration bonuses. The authors prove that PF-EFVI attains logarithmic regret under sufficient reward diversity, showing that disagreement among agents can shift the exploration burden from temporal to spatial aspects. This demonstrates that heterogeneous rewards can be leveraged as a structural asset to facilitate efficient collective learning.
AEM: Adaptive Entropy Modulation for Multi‑Turn Agentic Reinforcement Learning proposes a supervision‑free credit assignment technique that adjusts entropy dynamics at the response level during RL training. By aligning uncertainty estimation with the granularity of LLM agent actions and rescaling advantages based on a response‑level uncertainty proxy, AEM improves the exploration‑exploitation balance without relying on dense intermediate supervision. Experiments on ALFWorld, WebShop, and SWE‑bench‑Verified show consistent performance gains across models from 1.5B to 32B, including a 1.4% improvement in a state‑of‑the‑art software‑engineering RL framework.
The paper introduces EFXPLORER, a constrained exploration framework for cooperative multi-agent reinforcement learning that maximizes exploration objectives while preserving task performance. It uses an epigraph reformulation to create adaptive exploration budgets, separating intrinsic rewards from task objectives and regulating exploration through task feasibility. Additionally, a successor distance-based intrinsic reward is incorporated to promote diverse, temporally extended exploration, leading to improved performance and novel cooperative strategies on SMAX, VMAS, and MPE benchmarks.
arXiv:2608.21946v1 Announce Type: cross Abstract: Reinforcement learning with outcome-based objectives such as GRPO enables LLM-based agents to solve complex, long-horizon tasks, yet the reusable exp...
Recent breakthroughs of Reinforcement Learning (RL) have highlighted its potential for complex agentic Large Language Model (LLM) tasks. However, existing efforts largely focus on single-task settings, whereas real-world deployment necessitates a generalist agent capable of solving multiple tasks simultaneously.
arXiv:2607. 07178v1 Announce Type: cross Abstract: Recent breakthroughs of Reinforcement Learning (RL) have highlighted its potential for complex agentic Large Language Model (LLM) tasks.
The paper introduces Adaptive Mask Selection and Composition (AMSC), a method that estimates task similarity using non‑parametric Wasserstein embeddings from state‑action‑reward samples. AMSC selects and weights previously learned policies based on z‑score‑normalized sparsemax similarity scores, forming a prior for new tasks. Experiments on CT‑graph, MiniGrid, and Continual World show that AMSC improves mean performance and forward transfer while avoiding forgetting, highlighting the importance of selecting relevant prior knowledge and its layer‑specific composition.