The paper investigates how curriculum learning affects reinforcement learning in a minimal policy learning model. It shows that when rewards are dense, curricula offer only marginal benefits, but when rewards are sparse, curricula become essential for learning to occur. The study also highlights that competing objectives—minimizing effort versus maximizing reward—create distinct optima that curricula can help navigate by reducing reward sparsity and reshaping the reward landscape.
arXiv:2608. 05111v1 Announce Type: new Abstract: In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies.
The paper introduces SAGE, a framework that selectively queries a Vision‑Language Model (VLM) teacher only when the learner is uncertain, using the teacher’s suggestions to guide training and distill them into a lightweight reinforcement learning policy. SAGE weights teacher actions by environment‑derived advantages, allowing the policy to improve beyond the imperfect VLM. Experiments on sparse‑reward visual reasoning and navigation tasks show that the learned policies can act without VLM guidance at evaluation, reduce VLM usage during training, and sometimes outperform the teacher itself.
The paper introduces Bidirectional Voronoi-biased Exploration Curriculum (BVER), a method that expands start states from the goal and goals from the initial state simultaneously, guiding both toward each other to train a single goal-conditioned policy. Inspired by bidirectional RRT planning, BVER biases exploration toward unexplored task space and steers the two expansions together. Experiments on point-mass mazes, quadrupedal box climbing, and robot-arm ring-on-peg transfer show that BVER learns faster than other reference-free curricula, achieving high success rates and robustness without requiring demonstrations.
arXiv:2606. 29980v1 Announce Type: new Abstract: Zero-shot Transfer in Reinforcement Learning (RL) aims to train an agent that can generate optimal policies for any reward function, without additional learning at transfer time, while training only on reward-free trajectories.
arXiv:2608.30406v1 Announce Type: new Abstract: Goal-conditioned reinforcement learning struggles with long horizons when rewards are sparse. While a planner can provide subgoals to guide a low-level...
arXiv:2611.07130v1 Announce Type: new Abstract: Reinforcement learning has achieved strong performance in many tasks by letting agents learn from nothing more than a reward signal. However, in more c...
The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.
The paper investigates when intrinsic rewards effectively drive exploration in reinforcement learning. It introduces a formal criterion that evaluates policies based on the counterfactual information they acquire, comparing how well their histories can replace experience from alternative policies. Using a simple environment, the authors show that common intrinsic reward objectives—count-based, prediction-error, empowerment, and information-gain—can lead to Pareto-suboptimal exploration under this criterion, and they propose conditions and a new objective that better align with optimal exploration.
arXiv:2608. 07228v1 Announce Type: new Abstract: When a reinforcement learning agent cannot observe the full state, we usually blame its policies: it cannot see enough to represent a good one.
The paper studies repeated principal-agent games in a multi-armed bandit setting where the principal offers arm-specific incentives, the agent chooses an arm to maximize its own reward plus incentive, and the principal observes only the chosen arm and its own reward. Because the principal cannot see the agent’s reward, traditional estimation fails, so the authors develop an estimator based on the sequence of incentives and chosen arms, analogous to online inverse optimization. Using this estimator, they design a policy that achieves low (square‑root) regret and validate the approach with simulations for personalized incentive design in long‑term statin treatment.
arXiv:2608. 01425v1 Announce Type: cross Abstract: Training LLM-based multi-agent systems with multi-agent reinforcement learning is rapidly gaining traction, and a parallel line of work argues that such systems should be judged by their behavior, not only their reward.
arXiv:2606. 27180v1 Announce Type: cross Abstract: Sparse rewards are inherently challenging for reinforcement learning agents as they lack intermediate feedback to guide exploration and to correctly attribute the sparse success rewards to relevant parts of the trajectory.