arXiv AI By Guilin Zhang, Ziqi Tan, Wulan Guo, Kai Zhao, Hongyun Yang, Mei Luo, Qi Ning, Feng Yang

Budget Boundary Effects in Test-Time Mathematical Reasoning

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv AI
Sep 4

It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories

The paper investigates whether large language models’ reasoning traces truly contain early, informative signals or merely reflect budget and difficulty confounds. Using a restart‑controlled truncation probe, the authors compare continuation success rates against from‑scratch restart curves across 178 problem‑model pairs, finding that only one case shows prefix‑limited success and that continuing a model’s own prefix generally outperforms restarting. A difficulty‑controlled test and two generation‑free analyses reveal that early internal signals do not carry outcome information beyond a problem‑difficulty baseline, underscoring the need for proper counterfactual controls.

By Yigit Utku Bulut
arXiv Machine Learning
Oct 3

How Much Can Language Models Gain from Test-Time Computation?

The paper investigates how test‑time computation can enhance language models and at what cost, introducing the SELF‑POT benchmark to evaluate this across competition mathematics, competitive programming, and agentic workflows. SELF‑POT separates candidate coverage from final accuracy, tracks correctness transitions under revision, and measures protocol completion alongside task success. Using a unified budget rule, the study compares direct inference, parallel sampling, and self‑revision across five low‑cost reasoning models, revealing that selection rules and failure handling significantly influence gains and cost savings.

By Bangji Yang, Jingyuan Li, Jiajun Fan, Yi Evie Zhang, Ruihan Guo, Hongba Ma, Neil He, Chumeng Liang, Qinglong Zheng, Zhanghan Ni, Ge Liu
arXiv Computation and Language
Nov 4

When Does Adaptive Halting Win? Selection Rules Matter in Looped Language Models

The study investigates how different selection rules affect the performance of adaptive halting in looped language models, specifically comparing a trained stability probe with Ouro's native exit gate on a frozen Ouro-1.4B model. In a pre‑registered confirmatory experiment, the probe saved more of the four‑loop budget than the native gate, achieving a gross macro saving of +0.053 across three benchmarks, but a secondary point‑only analysis reversed this advantage. Additional comparisons showed no clear benefit over an untrained KL‑divergence baseline, and the probe performed poorly on an out‑of‑distribution benchmark, highlighting the importance of jointly evaluating halting signals and selection procedures.

By Mahesh Reddy Pagadala
arXiv Machine Learning
Oct 19

Sample, Mix, or Verify? Allocating Test-Time Compute under Imperfect Verifiers

The paper introduces SPLIT, a method that optimally allocates test‑time compute among sampling, model selection, and verification for language models and agents. Using an exact probability formula that accounts for task difficulty, shared blind spots, and verifier correlations, SPLIT is fitted via empirical Bayes on a small labeled pilot and then used to plan budgets for unseen tasks. Across seven diverse benchmarks, 26 models, and six verifier types, SPLIT achieves near‑optimal performance, outperforming rule‑based baselines by up to 2.5 percentage points and providing insights into when additional LLM judges reduce precision.

By Srinivasan Iyengar