The paper introduces a method to enhance large language model (LLM) exploration in Reinforcement Learning with Verifiable Rewards (RLVR) by guiding the target model with partial reasoning trajectories from smaller, weaker language models. This weak-model guidance disrupts over‑confidence, preserves generative diversity, and mitigates entropy collapse without extra fine‑tuning or complex reward designs. Experiments on mathematical benchmarks show consistent improvements over vanilla RLVR, especially as the number of allowed attempts ($k$) increases, indicating broader reasoning coverage.
By Xingyu Shen, Huishuai Zhang, Peng Li, Yinchun Wang, Dongyan Zhao
arXiv:2510. 13554v2 Announce Type: replace-cross Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps.
By Yang Li, Zhichen Dong, Yuhan Sun, Weixun Wang, Shaopan Xiong, Yijia Luo, Jiashun Liu, Han Lu, Jiamang Wang, Wenbo Su, Bo Zheng, Junchi Yan
arXiv:2608.30627v1 Announce Type: new
Abstract: As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token pred...
By Haoran Que, Jiajun Shi, Ting Huang, Renming Pang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Shen Yan, Wei Ye, Shikun Zhang
arXiv:2607. 19345v1 Announce Type: cross Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier.
By Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang
arXiv:2609.37119v1 Announce Type: cross
Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instabilit...
By Hongyang Li, Xiao Li, Caesar Wu, Said Mammar, Gr\'egoire Danoy, Pascal Bouvry
arXiv:2607. 18100v1 Announce Type: new Abstract: Extended reasoning has become standard for frontier Large Language Models (LLMs), yet the trajectories these models produce remain largely uncontrollable.
By Sheldon Yu, Tong Yu, Xunyi Jiang, Rohan Surana, Gagan Mundada, Sungchul Kim, Lina Yao, Julian McAuley, Junda Wu
arXiv:2508. 10123v3 Announce Type: replace-cross Abstract: Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT).
By Maxime Heuillet, Yufei Cui, Boxing Chen, Audrey Durand, Prasanna Parthasarathi
arXiv:2608.20359v1 Announce Type: new
Abstract: Large language models (LLMs) are deployed for increasingly complex tasks involving planning and multi-step decision making, but high-quality performanc...
By Ravisri Valluri, Tung Nguyen, Aditya Grover
arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.
By Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Liwei Qian, Xin Pei, Jizhou Huang
arXiv:2507.21931v2 Announce Type: replace-cross
Abstract: Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks....
By Carel van Niekerk, Renato Vukovic, Benjamin Ruppik, Hsien-chin Lin, Shutong Feng, Milica Ga\v{s}i\'c
arXiv:2606. 14142v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly adopted as backbones for Generative Recommendation (GR), promising access to pretrained world knowledge.
By Yinhan He, Liam Collins, Bhuvesh Kumar, Jundong Li, Neil Shah, Donald Loveland
The paper demonstrates that fine‑tuning reasoning models to predict their own confidence at intermediate steps—using only 600 self‑supervised examples—substantially improves inference efficiency. Without adding any explicit stopping or length penalties, the models generate up to 25 % fewer tokens while maintaining accuracy on mathematical, scientific, and coding benchmarks across several architectures. The study finds that confidence supervision preserves the models’ high‑level reasoning structure rather than merely suppressing specific behaviors.
By Parsa Hosseini, Akasha Tigalappanavara, Sumit Nawathe, Chenrui Fan, Sourya Basu, Genta Indra Winata, Anirban Das, Soheil Feizi, Nima Chitsazan