The paper introduces SAGE, a framework that selectively queries a Vision‑Language Model (VLM) teacher only when the learner is uncertain, using the teacher’s suggestions to guide training and distill them into a lightweight reinforcement learning policy. SAGE weights teacher actions by environment‑derived advantages, allowing the policy to improve beyond the imperfect VLM. Experiments on sparse‑reward visual reasoning and navigation tasks show that the learned policies can act without VLM guidance at evaluation, reduce VLM usage during training, and sometimes outperform the teacher itself.
arXiv:2606. 27180v1 Announce Type: cross Abstract: Sparse rewards are inherently challenging for reinforcement learning agents as they lack intermediate feedback to guide exploration and to correctly attribute the sparse success rewards to relevant parts of the trajectory.
arXiv:2607.08837v4 Announce Type: replace-cross Abstract: Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers. Standard methods inject...
The paper introduces Instruct-to-Act, a system that decouples planning and control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them at high frequency, trained via relabeling rollouts with synthetic instructions and joint optimization of behavior cloning, reward, and world‑model objectives. Across seven embodied environments—including multi‑agent settings—this approach outperforms controller‑only and direct VLM action methods, maintains fast control, and allows swapping pretrained VLM planners without fine‑tuning, achieving competitive results with strong baselines on most tasks.
arXiv:2604.00055v2 Announce Type: replace-cross Abstract: Existing robotic foundation policies are trained primarily via large-scale imitation learning. While such models demonstrate strong capabilit...
arXiv:2607. 08837v1 Announce Type: cross Abstract: Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers.
The paper introduces MENTOR, a multi‑agent framework that provides on‑policy pedagogical intervention to address zero‑reward collapse in reinforcement learning from verifiable rewards. MENTOR diagnoses failures in real time and generates targeted prompts to guide the learning agent, integrating seamlessly with existing policy optimization pipelines such as GRPO. Experiments on math and code generation tasks show that MENTOR significantly reduces collapse, achieving up to 78.3% relative gains on Llama‑3.2‑3B‑Instruct.
The paper introduces Dwell, a model‑based reinforcement learning framework that treats a pretrained vision‑language model (VLM) as a recurrent latent transition operator. Dwell enables the VLM to predict, adjust, and act within its own latent space, allowing the agent to learn from real interactions and latent imagination without reconstructing full futures. Experiments across five interactive visual environments show that Dwell outperforms other trainable methods while significantly reducing computational demands for imagination.
The paper introduces Instruct-to-Act, a system that decouples high‑level planning from low‑latency control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them autonomously at high frequency. Experiments across seven embodied environments, including multi‑agent settings, show that this approach outperforms both controller‑only and direct VLM action‑generation methods, maintains fast control, and allows swapping in different pretrained VLM planners without fine‑tuning.
arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.
arXiv:2608.24696v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) and on-policy distillation (OPD) have become two widely adopted paradigms for post-training lar...
The paper investigates how curriculum learning affects reinforcement learning in a minimal policy learning model. It shows that when rewards are dense, curricula offer only marginal benefits, but when rewards are sparse, curricula become essential for learning to occur. The study also highlights that competing objectives—minimizing effort versus maximizing reward—create distinct optima that curricula can help navigate by reducing reward sparsity and reshaping the reward landscape.
arXiv:2512. 03438v3 Announce Type: replace Abstract: Agentic reasoning models trained with multimodal reinforcement learning (MMRL) have become increasingly capable, yet they are almost universally optimized using sparse, outcome-based rewards computed based on the final answers.