arXiv AI
Prompt-Driven Exploration: Language as an Exploration Space for VLA Reinforcement Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 13
Prompt-Driven Exploration
arXiv:2607. 08837v1 Announce Type: cross Abstract: Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers.
arXiv AI
Sep 15
Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training
The paper introduces an exploration-guided prompt scaffolding framework for multimodal large language models, dynamically adjusting the prompt distribution during reinforcement learning post-training. It uses an Exploration Potential Score (EPS) derived from KL-regularized policy improvement to assess prompt utility without extra overhead, and a teacher model rewrites low-utility prompts to preserve intent while improving informativeness. Experiments on Geo3K, MMK12, MathVision, and MMMU-Pro show consistent performance gains, up to 9.7% in-domain and over 11% on out-of-distribution benchmarks.
arXiv Machine Learning
Aug 27
Demystifying Reinforcement Learning Post-Training of Language Models
The paper "Demystifying Reinforcement Learning Post-Training of Language Models" investigates how reinforcement learning (RL) post‑training enhances large language models (LLMs) for tasks such as reasoning, math, and coding. By isolating RL components in a controlled setting, the authors analyze how the base model’s prior distribution, reward granularity, prompt diversity, and model scale influence outcomes, using policy entropy to compare pre‑training, supervised fine‑tuning (SFT), and RL stages. The study clarifies the role of spurious rewards, the importance of the base model’s probability mass on desired behaviors, and how these factors interact to determine post‑training success, offering a practical primer for NLP researchers. "whyItMatters":"The work provides a clearer understanding of RL post‑training mechanics, helping researchers and practitioners effectively apply RL to improve LLM capabilities."
arXiv Machine Learning
Oct 8
Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization
arXiv:2610.09943v1 Announce Type: cross Abstract: Reinforcement learning (RL) fine-tuning improves vision-language-action (VLA) policies through closed-loop experience, yet generalization beyond the...
arXiv AI
Oct 12
Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits
The paper introduces Rollout-Time Intervention (RTI), a method that uses reward-side evidence during rollouts to transform unsuccessful trajectories into successful ones, thereby providing learning signals for tasks beyond a policy’s intrinsic exploration limits. RTI filters out trajectories with excessive distributional shifts, allowing these off-policy interventions to guide policy optimization effectively. Experiments on five benchmarks show that RTI consistently outperforms existing RL baselines, enabling post-trained policies to approach the performance of stronger large language models and surpass their original capability boundaries.
arXiv Machine Learning
Jun 4
Policy Improvement Reinforcement Learning
arXiv:2604. 00860v3 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a central post-training paradigm for improving the reasoning capabilities of large language models.