The paper investigates whether large language models’ reasoning traces truly contain early, informative signals or merely reflect budget and difficulty confounds. Using a restart‑controlled truncation probe, the authors compare continuation success rates against from‑scratch restart curves across 178 problem‑model pairs, finding that only one case shows prefix‑limited success and that continuing a model’s own prefix generally outperforms restarting. A difficulty‑controlled test and two generation‑free analyses reveal that early internal signals do not carry outcome information beyond a problem‑difficulty baseline, underscoring the need for proper counterfactual controls.
By Yigit Utku Bulut
arXiv:2608. 04160v1 Announce Type: cross Abstract: Multilingual evaluations report accuracy at a single output-token cap, but languages need different numbers of tokens to express the same content, so the cap is a hidden experimental variable.
By Ankit Goyal, Jaideep Ray
The paper investigates how test‑time computation can enhance language models and at what cost, introducing the SELF‑POT benchmark to evaluate this across competition mathematics, competitive programming, and agentic workflows. SELF‑POT separates candidate coverage from final accuracy, tracks correctness transitions under revision, and measures protocol completion alongside task success. Using a unified budget rule, the study compares direct inference, parallel sampling, and self‑revision across five low‑cost reasoning models, revealing that selection rules and failure handling significantly influence gains and cost savings.
By Bangji Yang, Jingyuan Li, Jiajun Fan, Yi Evie Zhang, Ruihan Guo, Hongba Ma, Neil He, Chumeng Liang, Qinglong Zheng, Zhanghan Ni, Ge Liu
The study investigates how different selection rules affect the performance of adaptive halting in looped language models, specifically comparing a trained stability probe with Ouro's native exit gate on a frozen Ouro-1.4B model. In a pre‑registered confirmatory experiment, the probe saved more of the four‑loop budget than the native gate, achieving a gross macro saving of +0.053 across three benchmarks, but a secondary point‑only analysis reversed this advantage. Additional comparisons showed no clear benefit over an untrained KL‑divergence baseline, and the probe performed poorly on an out‑of‑distribution benchmark, highlighting the importance of jointly evaluating halting signals and selection procedures.
By Mahesh Reddy Pagadala
arXiv:2608. 03401v1 Announce Type: cross Abstract: Large language models often reason at length before answering, increasing cost and latency.
By Francesca Carlon, Vincent Ginis, Andres Algaba
The paper introduces SPLIT, a method that optimally allocates test‑time compute among sampling, model selection, and verification for language models and agents. Using an exact probability formula that accounts for task difficulty, shared blind spots, and verifier correlations, SPLIT is fitted via empirical Bayes on a small labeled pilot and then used to plan budgets for unseen tasks. Across seven diverse benchmarks, 26 models, and six verifier types, SPLIT achieves near‑optimal performance, outperforming rule‑based baselines by up to 2.5 percentage points and providing insights into when additional LLM judges reduce precision.
By Srinivasan Iyengar
The paper investigates whether providing candidate solutions during test‑time aggregation improves or harms accuracy compared to a fresh solve that does not use any candidates. Using Qwen3‑4B on AIME‑2025 and HMMT‑2025, the authors find that conditioning on multiple correct candidates boosts accuracy (+0.290), while conditioning on an all‑wrong candidate pool reduces accuracy (−0.123); the effect for a single correct candidate remains unclear. The study also explores structured interventions and placebo controls, but the underlying mechanisms of these effects are not resolved.
By Guiv Farmanfarmaian
arXiv:2610.25768v2 Announce Type: replace
Abstract: Large language models (LLMs) often generate more reasoning tokens than needed. Limiting the reasoning budget reduces cost, but forcing an answer fr...
By Zhongyin Wang, Zhaoqian Xue, Junting Mu, Haozheng Liu, Ye Yuan, Haolun Wu, Lingyao Li
arXiv:2608. 07303v1 Announce Type: new Abstract: Comparisons between AutoML systems at short time budgets -- tens of seconds rather than hours -- are common in tool READMEs and workshop papers, and they are easy to get wrong.
By Guilin Zhang, Kai Zhao
The paper evaluates partial‑prompt completion tests—where a model is given the first part of a benchmark problem and checked for reproducing the rest—on seven open‑weight models using MATH‑500 and recent AIME 2025/2026 problems as true negatives. It finds that memorization flags are highly sensitive to the amount of text left: problems with fewer than eight remaining words are flagged far more often than those with 25 or more, and most ROUGE‑based flags arise from short leftovers. On true negatives, even a high ROUGE‑L threshold flags up to 6.7% of problems at a 60% prefix, and exact‑match flags 8.6% at an 80% prefix, while models distilled from Qwen2.5‑Math show almost no recall of contamination. The authors recommend length‑stratified reporting, limiting prefixes to 60%, using style‑matched post‑cutoff controls, and cross‑family calibration to reduce false positives.
By Abhishek Anand
Test-time scaling can improve LLM reasoning, but existing approaches typically adapt generation or apply answer selection after a predetermined generation schedule. We study within-instance allocation...
arXiv:2606. 19808v1 Announce Type: new Abstract: Test-time reasoning is increasingly used as a serving-time control knob, but extra reasoning is not uniformly valuable: it can repair failed attempts, waste compute on already-correct answers, or introduce harmful answer changes.
By Sajib Acharjee Dip, Dawei Zhou, Liqing Zhang