arXiv:2609.38104v1 Announce Type: new
Abstract: Power-sharpened sampling is an inference-time alternative to reinforcement-learning (RL) post-training for enhancing reasoning in large language models...
By Panagiotis Theodoropoulos, Nan Jiang, Xintong Duan, Ali Hasan, Yuriy Nevmyvaka, Evangelos A. Theodorou, Wei Deng
The paper introduces a method to enhance large language model (LLM) exploration in Reinforcement Learning with Verifiable Rewards (RLVR) by guiding the target model with partial reasoning trajectories from smaller, weaker language models. This weak-model guidance disrupts over‑confidence, preserves generative diversity, and mitigates entropy collapse without extra fine‑tuning or complex reward designs. Experiments on mathematical benchmarks show consistent improvements over vanilla RLVR, especially as the number of allowed attempts ($k$) increases, indicating broader reasoning coverage.
By Xingyu Shen, Huishuai Zhang, Peng Li, Yinchun Wang, Dongyan Zhao
arXiv:2607. 06720v1 Announce Type: new Abstract: Training large language models (LLMs) with extended reasoning has enabled in-context search, in which models iteratively generate, critique, and revise solution attempts.
By Yotam Wolf, Noam Wies, Amnon Shashua
arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.
By Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Liwei Qian, Xin Pei, Jizhou Huang
The paper introduces CARE, a contrastive accuracy reward estimation method that adaptively adjusts reasoning length for large language models. By comparing beneficial length adjustments from online sampled responses, CARE applies adaptive length rewards within Group Relative Policy Optimization without extra hyperparameters or inference cost. Experiments on multiple reasoning benchmarks show that CARE improves Pass@1 by up to 4% while reducing reasoning length by 37%, achieving higher token efficiency.
By Zhengdong He, Yunfan Zhou, Jianguo Yao, Haibing Guan, Xijun Li
The paper introduces Exploration-Distillation (ExpDis), a method that separates exploration from optimization in reinforcement learning with verifiable rewards (RLVR) for language models. ExpDis trains explorer policies with a novelty bonus, filters their outputs for correctness, and distills the results into a student policy that is optimized without the novelty bonus. This iterative process enables aggressive exploration while preserving model quality, outperforming DAPO on multiple mathematical reasoning benchmarks and improving pass@$k$ scaling.
By Saif Punjwani, Micah Goldblum
Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identical rollout rewards consume generation budget without effective learning signals. Pre-rollout prompt selection can reduce this waste by screening prompts before rollout generation.
arXiv:2607.28077v2 Announce Type: replace
Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identica...
By Shuang Liang, Haoyang Zhou, Yifan Gong, Guowei Wang, Xiting Wang
arXiv:2605.30327v2 Announce Type: replace-cross
Abstract: Frontier reasoning models are produced by post-training base language models with reinforcement learning. Recent work has challenged this by...
By Felix Zhou, Anay Mehrotra, Quanquan C. Liu
arXiv:2510. 11686v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) promises to expand the capabilities of language models, but it is unclear if current RL techniques promote the discovery of novel behaviors, or simply sharpen those already present in the base model.
By Jens Tuyls, Dylan J. Foster, Akshay Krishnamurthy, Jordan T. Ash
arXiv:2606. 17024v1 Announce Type: new Abstract: Sparse reward reinforcement learning (RL) has become a standard tool for improving LLM reasoning, but its success depends critically on the coverage present in the base model.
By Violet Xiang, Amrith Setlur, Chase Blagden, Nick Haber, Aviral Kumar
ReAPO: Retry-Aware Policy Optimization for Efficient Exploration in LLM Reasoning proposes a new approach to improve language‑model reasoning by addressing the inefficiencies of independent stochastic rollouts. By framing credit assignment as a Stochastic Shortest Path problem, ReAPO modifies GRPO to allocate more generation to failed trajectories, reinforcing successful retries without rewarding failures. Experiments on Qwen3 and OLMo‑2 models across eight mathematical and scientific benchmarks show that ReAPO outperforms GRPO and other baselines, achieving higher first‑attempt accuracy and reducing generated tokens needed to reach peak performance.
By Nghi Dao, Calder Morton-Ferguson, Noah Golowich, Adam Block