arXiv:2608. 20256v1 Announce Type: new Abstract: Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones.
By Gijs Kassenaar, Zhao Yang, Vincent Fran\c{c}ois-Lavet
When2Think introduces a post‑training framework that dynamically allocates reasoning depth in Large Reasoning Models based on instance difficulty. The method uses Instance‑level Difficulty‑Aware Control (IDAC) to shape rewards with pre‑computed accuracy and token usage statistics, enabling stable, critic‑free optimization without learned reward models. Experiments on mathematical benchmarks show that When2Think improves accuracy‑efficiency trade‑offs, achieving higher Pass@3 scores while reducing token usage compared to baseline models.
By Jaejun Shim, HyunJin Kim, Young Jin Kim, JinYeong Bak
The paper introduces PORA, a method that teaches large language model agents to decide when to allocate more reasoning steps versus fewer, based on the expected downstream benefit. PORA trains by running each state twice—once with short reasoning and once with long reasoning—and learns to predict the value of extended reasoning. During inference, a controller uses this prediction and remaining budget information to choose the appropriate reasoning length without running both options, improving performance across diverse tasks.
By Bo Yan, Qi Zhang, Zhen Tan, Tianlong Chen, Song Wang
arXiv:2510.27042v3 Announce Type: replace
Abstract: Increasing the thinking budget of AI models can significantly improve accuracy, but not all questions warrant the same amount of reasoning. Users m...
By Michael Kleinman, Matthew Trager, Alessandro Achille, Wei Xia, Stefano Soatto
The paper introduces AnytimeReasoner, a framework that optimizes large language models for anytime reasoning by truncating the reasoning process to fit sampled token budgets and requiring the model to summarize the answer for verification. This approach adds verifiable dense rewards, improving credit assignment in reinforcement learning. The authors also propose Budget Relative Policy Optimization (BRPO), a variance reduction technique that enhances learning robustness and efficiency, and demonstrate that their method outperforms GRPO on mathematical reasoning tasks across various budgets and prior distributions.
By Penghui Qi, Zichen Liu, Tianyu Pang, Chao Du, Wee Sun Lee, Min Lin
arXiv:2510.01581v2 Announce Type: replace-cross
Abstract: Recent thinking models are capable of solving complex reasoning tasks by scaling test-time compute, but this scaling should be allocated in l...
By Joykirat Singh, Justin Chih-Yao Chen, Archiki Prasad, Elias Stengel-Eskin, Akshay Nambi, Mohit Bansal
The paper proposes a new framework for efficient chain‑of‑thought (CoT) prompting called the Conditional Information Bottleneck (CIB). By treating reasoning as a lossy compression problem, CIB models the reasoning trace as a computational bridge that contains only the information about the response not directly available from the prompt, thereby overcoming a theoretical gap in naive Information Bottleneck approaches. The resulting reinforcement‑learning objective maximizes task reward while compressing completions under a semantic prior based on token surprisal, leading to reduced redundancy, preserved fluency and logic, and improved accuracy across models and tasks.
By Fabio Valerio Massoli, Andrey Kuzmin, Arash Behboodi
arXiv:2511. 02130v2 Announce Type: replace-cross Abstract: We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens.
By Renos Zabounidis, Aditya Golatkar, Michael Kleinman, Alessandro Achille, Wei Xia, Stefano Soatto
arXiv:2606. 05464v1 Announce Type: new Abstract: Verifiable reward training has improved mathematical and coding reasoning, but these domains capture only part of step-by-step decision making.
By Nicol\'as Astorga, Nabeel Seedat, Mihaela van der Schaar
arXiv:2601.03822v2 Announce Type: replace
Abstract: Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computa...
By Muyang Zhao, Chunhao Tian, Qi Qi, Hao Sun
Large Reasoning Models (LRMs) excel on complex tasks through long chain-of-thought (CoT) reasoning, but their lengthy intermediate steps cause severe overthinking that inflates inference cost. KV-cache compression is a common solution, yet existing reasoning-oriented methods apply a uniform policy across the trajectory and judge compression only by what it removes from the cache.
The paper introduces a new reward, GRPO, that encourages large reasoning models (LRMs) to efficiently determine whether a task is solvable before generating a full chain of thought. Fine‑tuning 4B LRMs with this reward improves their ability to abstain from answering unanswerable prompts by an average of 12.8% while producing 44% shorter chains of thought. The approach also preserves the models’ overall answering performance.
By Polina Tsvilodub, Max H\"oth, Michael Franke, Bj\"orn Deiseroth, Carina Kauf