arXiv:2609.37119v1 Announce Type: cross Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instabilit...
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
The paper introduces POISE, a reinforcement learning algorithm that uses a model’s internal states as a value estimator to reduce variance in reinforcement learning with verifiable rewards (RLVR). By employing a lightweight probe that reads internal signals during the forward pass, POISE predicts baselines online and uses a cross‑rollout construction to keep gradients unbiased. Experiments on Qwen3‑4B and OLMo3‑7B‑Instruct‑DPO across six domains show POISE outperforms existing RLVR baselines, offering more stable training and a value model that generalizes across tasks and scales with the policy.
arXiv:2606. 30420v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is a powerful paradigm for improving the reasoning capabilities of large language models (LLMs).
arXiv:2610.15840v1 Announce Type: cross Abstract: Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assign...
The paper introduces Single‑Rollout Pass@k Policy Optimization (SR‑PPO), a reinforcement learning method that uses only one rollout per prompt to train language‑model agents. SR‑PPO combines Monte Carlo token‑level advantage estimation with Pass@k gradient reweighting and an online critic for baseline and weight estimation. Experiments show that SR‑PPO achieves stable learning and performance comparable to traditional Group Relative Policy Optimization (GRPO) on reasoning and logic benchmarks, even when off‑policy drift degrades standard Pass@1 PPO.
ReAPO: Retry-Aware Policy Optimization for Efficient Exploration in LLM Reasoning proposes a new approach to improve language‑model reasoning by addressing the inefficiencies of independent stochastic rollouts. By framing credit assignment as a Stochastic Shortest Path problem, ReAPO modifies GRPO to allocate more generation to failed trajectories, reinforcing successful retries without rewarding failures. Experiments on Qwen3 and OLMo‑2 models across eight mathematical and scientific benchmarks show that ReAPO outperforms GRPO and other baselines, achieving higher first‑attempt accuracy and reducing generated tokens needed to reach peak performance.
arXiv:2606. 07950v1 Announce Type: new Abstract: RL with verifiable rewards can substantially improve LLM reasoning, yet standard GRPO-style training often treats easy, hard, and learnable questions alike through uniform sampling and weighting, leading to inefficient compute allocation.
arXiv:2603. 25184v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become essential for post-training large language models (LLMs) in reasoning tasks.
arXiv:2508. 10123v3 Announce Type: replace-cross Abstract: Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT).
arXiv:2605.27293v2 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards has become a standard recipe for improving the reasoning abilities of large language models. Existin...
arXiv:2609.37825v1 Announce Type: cross Abstract: Assigning credit to intermediate steps remains a central challenge in training Large Language Models (LLMs) on multi-step reasoning tasks with sparse...