arXiv Machine Learning
ReAPO: Retry-Aware Policy Optimization for Efficient Exploration in LLM Reasoning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 2
RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Jun 25
Learning with a Single Rollout via Monte Carlo Pass@k Critic
arXiv:2606. 25451v1 Announce Type: new Abstract: Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive.
arXiv AI
Oct 15
Double-Rollout Reinforcement Learning with Sampling-based Reward Estimation
arXiv:2610.16593v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) implemented with Group Relative Policy Optimization (GRPO) has driven substantial progress in tra...
arXiv AI
Sep 30
Diversifying RLVR Rollouts via First-Token Exploration
arXiv:2605.28295v2 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) trains reasoning models without labeled trajectories, using groups of verifier-scored rollout...
arXiv AI
Jun 17
Beyond the Sampled Token: Preserving Candidate Support in RLVR
arXiv:2510. 14807v3 Announce Type: replace Abstract: We revisit exploration collapse in reinforcement learning with verifiable rewards (RLVR), from the perspective of the \emph{candidate distribution} for next-token prediction.
arXiv Machine Learning
Oct 8
Decoupling Exploration from Optimization in RLVR
The paper introduces Exploration-Distillation (ExpDis), a method that separates exploration from optimization in reinforcement learning with verifiable rewards (RLVR) for language models. ExpDis trains explorer policies with a novelty bonus, filters their outputs for correctness, and distills the results into a student policy that is optimized without the novelty bonus. This iterative process enables aggressive exploration while preserving model quality, outperforming DAPO on multiple mathematical reasoning benchmarks and improving pass@$k$ scaling.