arXiv:2510. 14807v3 Announce Type: replace Abstract: We revisit exploration collapse in reinforcement learning with verifiable rewards (RLVR), from the perspective of the \emph{candidate distribution} for next-token prediction.
By Ruotian Peng, Yi Ren, Zhouliang Yu, Weiyang Liu, Yandong Wen
The paper introduces DATPO, a Difficulty‑Adaptive Sentence‑entropy‑guided Tree‑structured Policy Optimization method designed to improve reasoning coverage in Reinforcement Learning with Verifiable Rewards (RLVR). It builds on three design principles: adaptive difficulty rollouts, tree‑based rollouts, and sentence‑entropy‑guided forking to enhance semantic diversity. Experiments on mathematical reasoning benchmarks show that DATPO outperforms existing baselines, particularly in pass@k, leading to better test‑time scaling performance.
By Youngjun Yu, Sanghwan Jang, Hwanjo Yu
arXiv:2610.27012v1 Announce Type: cross
Abstract: Reinforcement learning with verifiable rewards (RLVR) trains a language model by sampling responses, scoring them with an automatic verifier, and upd...
By Yunan Zeng, Jinhao Jing, Xi Cheng, Yuwang Wang
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
By Yixiu Mao, Yun Qu, Qi Wang, Heming Zou, Xiangyang Ji
ReAPO: Retry-Aware Policy Optimization for Efficient Exploration in LLM Reasoning proposes a new approach to improve language‑model reasoning by addressing the inefficiencies of independent stochastic rollouts. By framing credit assignment as a Stochastic Shortest Path problem, ReAPO modifies GRPO to allocate more generation to failed trajectories, reinforcing successful retries without rewarding failures. Experiments on Qwen3 and OLMo‑2 models across eight mathematical and scientific benchmarks show that ReAPO outperforms GRPO and other baselines, achieving higher first‑attempt accuracy and reducing generated tokens needed to reach peak performance.
By Nghi Dao, Calder Morton-Ferguson, Noah Golowich, Adam Block
The paper introduces a method to enhance large language model (LLM) exploration in Reinforcement Learning with Verifiable Rewards (RLVR) by guiding the target model with partial reasoning trajectories from smaller, weaker language models. This weak-model guidance disrupts over‑confidence, preserves generative diversity, and mitigates entropy collapse without extra fine‑tuning or complex reward designs. Experiments on mathematical benchmarks show consistent improvements over vanilla RLVR, especially as the number of allowed attempts ($k$) increases, indicating broader reasoning coverage.
By Xingyu Shen, Huishuai Zhang, Peng Li, Yinchun Wang, Dongyan Zhao
arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.
By Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Liwei Qian, Xin Pei, Jizhou Huang
arXiv:2606. 15455v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a key approach for enhancing the reasoning abilities of large language models.
By Suqin Yuan, Jinkun Chen, Jiyang Zheng, Muyang Li, Lei Feng, Dadong Wang, Tao Xiang, Tongliang Liu, Bo An
Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identical rollout rewards consume generation budget without effective learning signals. Pre-rollout prompt selection can reduce this waste by screening prompts before rollout generation.
arXiv:2607.28077v2 Announce Type: replace
Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identica...
By Shuang Liang, Haoyang Zhou, Yifan Gong, Guowei Wang, Xiting Wang
The paper introduces EquivAug, a data augmentation framework that applies symbolic equivalence transformations to chain-of-thought (CoT) solutions, generating diverse yet provably correct reasoning paths for supervised fine-tuning (SFT). By converting CoT solutions into formal symbolic representations, applying equivalence transformations, and reconstructing natural-language explanations, EquivAug produces training data that varies in reasoning steps rather than surface-level lexical differences. Experiments on GSM8K and FOLIO show that EquivAug consistently achieves the highest reasoning path diversity after SFT and the strongest accuracy after GRPO, establishing a positive correlation between CoT diversity during SFT and downstream reinforcement learning with verifiable rewards (RLVR) performance.
By Xiaoou Liu, Tiejin Chen, Hanyang Chen, Yuan Ling, Hua Wei
arXiv:2507.14843v5 Announce Type: replace
Abstract: Recent advances highlight Reinforcement Learning with Verifiable Rewards (RLVR) as a promising approach to enhancing LLM capabilities. However, it...
By Fang Wu, Weihao Xuan, Danny Xing, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, Yejin Choi