arXiv Machine Learning

Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning

arXiv:2511. 02130v2 Announce Type: replace-cross Abstract: We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens.

arXiv AI
Jul 2

CAT: Confidence-Adaptive Thinking for Efficient Reasoning of Large Reasoning Models

arXiv:2607. 00862v1 Announce Type: cross Abstract: Large Reasoning Models (LRMs) have achieved remarkable success on complex tasks by leveraging long chain-of-thought (CoT) trajectories, yet they frequently exhibit overthinking on simple queries, resulting in significant token overhead and reduced inference efficiency.

By Qizhi Jiang, Shuo Wang, Pei Ke, Yuhang Song, Ke Qin
arXiv AI
6d ago

Nice Fold or Hero Call: Learning Budget-Efficient Thinking under Policy-Dependent Solvability

The paper introduces Budget‑Efficient Thinking (BET), a two‑stage framework that treats adaptive reasoning as a computational investment, aligning solve‑or‑fold decisions with expected return rather than perceived difficulty. BET learns three distinct behaviors: concise short solves for easy queries, early abstention (nice fold) when further reasoning is unlikely to pay off, and allocating sufficient compute (hero call) for hard‑but‑solvable questions. Experiments on seven benchmarks with three base models show BET cuts reasoning tokens by 54% while boosting accuracy by up to 3.2%, and it transfers effectively to scientific QA and logical reasoning tasks.

By Zhaomeng Zhou, Lan Zhang, Junyang Wang, Mu Yuan, Songlin Liu, Tingzhao Li, Yiqing Hu, Yumeng Zhao
arXiv AI
Sep 18

When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models

When2Think introduces a post‑training framework that dynamically allocates reasoning depth in Large Reasoning Models based on instance difficulty. The method uses Instance‑level Difficulty‑Aware Control (IDAC) to shape rewards with pre‑computed accuracy and token usage statistics, enabling stable, critic‑free optimization without learned reward models. Experiments on mathematical benchmarks show that When2Think improves accuracy‑efficiency trade‑offs, achieving higher Pass@3 scores while reducing token usage compared to baseline models.

By Jaejun Shim, HyunJin Kim, Young Jin Kim, JinYeong Bak
arXiv AI
Aug 21

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

arXiv:2608. 20256v1 Announce Type: new Abstract: Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones.

By Gijs Kassenaar, Zhao Yang, Vincent Fran\c{c}ois-Lavet
arXiv AI
Sep 25

To Think or Not to Think: Allocating Reasoning Where It Helps

The paper introduces CARE, a contrastive accuracy reward estimation method that adaptively adjusts reasoning length for large language models. By comparing beneficial length adjustments from online sampled responses, CARE applies adaptive length rewards within Group Relative Policy Optimization without extra hyperparameters or inference cost. Experiments on multiple reasoning benchmarks show that CARE improves Pass@1 by up to 4% while reducing reasoning length by 37%, achieving higher token efficiency.

By Zhengdong He, Yunfan Zhou, Jianguo Yao, Haibing Guan, Xijun Li