The paper discusses how the same computational task can require different reasoning strategies depending on the order in which evidence arrives, introducing the concept of an "information‑slack dilemma." It argues that early computation may be useful only if its benefits outweigh the costs of later verification, invalidation, and recovery, and proposes a research agenda focused on selective recovery and predictive policies. The authors emphasize evaluating these approaches by separating early‑execution effects, deployment value versus full‑input alternatives, and the added value of predictive policies while considering shared‑resource costs.
By Xiaotian Zhang (Trooly.AI)
arXiv:2609.39334v1 Announce Type: cross
Abstract: Test-time scaling has recently emerged as a powerful approach for improving LLM reasoning by allocating additional computation during inference, subs...
By Jinwoo Jeong (Korea University), Woohyung Choi (Korea University), Myeongjae Jeon (POSTECH), Jeongseob Ahn (Korea University)
The paper investigates how explicit reasoning in language models can lead to a "reasoning‑trace overflow," where the model spends most of its 32k‑token output budget on reasoning rather than producing code. Across seven models and six code benchmarks, overflow is rare on simple tasks but common on harder benchmarks, often due to repetitive reasoning loops. The study shows that disabling reasoning can sometimes be competitive, while prompt steering and budgeted transitions can mitigate overflow and improve token efficiency.
By Lukas Twist, Helen Yannakoudakis, Jie M. Zhang
TRACES (Tagging Reasoning Steps for Adaptive Cost‑Efficient Early‑Stopping) is a lightweight framework that tags reasoning steps of large‑language models in real time, enabling adaptive, cost‑efficient early stopping during inference. By monitoring the types of steps generated, the method identifies when models shift their reasoning after arriving at a correct answer, allowing for interpretable stopping criteria. Experiments on mathematical reasoning benchmarks (MATH500, GSM8K, AIME) and knowledge benchmarks (MMLU, GPQA) show token reductions of 20–50% while preserving accuracy, with more conservative thresholds needed for harder tasks such as BeyondAIME and IMO AnswerBench.
By Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo, John D. Kelleher
arXiv:2610.18090v1 Announce Type: new
Abstract: Ambiguous or incomplete task specifications complicate both execution and clarification for language-model agents. Yet agents' responses to these infor...
By Kaiyu Huang, Yiting Li, Angyang Gu, Pinjia He, Minchen Yu, Qingjiang Shi
The paper introduces EasyProver, a Lean theorem prover that uses recursive agents, isolated contexts, asynchronous search, parallel tactic probing, and an elastic elaboration service. It achieves near‑saturation results on several benchmark suites, proving 669/672 PutnamBench problems, all six IMO 2026 formalizations, 597/600 NTP4VC verification conditions, and all 189 VERINA tasks. Additionally, the authors present Aegis, a pipeline that transforms competitive programming problems into harder verification tasks by generating safe Rust solutions, translating them to Lean, and producing 2,045 Rust‑Lean program pairs with 11,533 candidate specifications, enabling new benchmarks for provers and reinforcement learning.
By Debargha Ganguly, Divya Raghunathan, Dejan Jovanovi\'c, Vikash Singh, Soonho Kong, Aaron Tomb, Aman Goel, Ferhat Erata, Sam Bayless
The paper investigates whether an elicitation agent (EA) can improve the quality of Lean specifications generated by large language models (LLMs) through interactive clarification. The EA decomposes a task, identifies unresolved decisions, asks a limited number of clarification questions, and refines the natural‑language specification before an LLM formalizer translates it into Lean. Experiments on 189 Verina Lean 4 tasks show that cross‑model pipelines (e.g., Sonnet 4.6 → GPT‑5.6) can achieve up to a 65.6 % strict pass@1 rate, whereas same‑model pipelines do not consistently improve accuracy.
By Mowafak Allaham, Aditya Vempaty, Ashish Jagmohan, Prasenjit Dey, Ravi Kokku
arXiv:2609. 18366v1 Announce Type: new Abstract: Reliable agent evaluation is complicated by automatic harness optimization, which repeatedly uses a released benchmark $B_{\mathrm{rel}}$ to guide a Proposer that edits prompts, memory, retrieval, tools, and control code around a fixed target agent.
By Guojun Zhu, Xunheng Huang, Peng Yin, Jiahui Xie, Sanguo Zhang, Doudou Zhou
arXiv:2611.07194v1 Announce Type: cross
Abstract: Coding agents achieve high success rates on existing benchmarks for Verus, a verifier for Rust programs. These benchmarks typically provide the auxil...
By \'Alvaro F. Silva, Martim Afonso, Nuno Saavedra, Emily First, Alexandra Mendes, Jo\~ao F. Ferreira
arXiv:2606. 18733v1 Announce Type: cross Abstract: Realistic coding-agent benchmarks often replay public GitHub issues and pull requests, making them vulnerable to overlap with model pretraining, fine-tuning, synthetic-data generation, or benchmark-driven model selection.
By Qiao Zhao, JianYing Qu, Jun Zhang, Yehua Yang, Hanwen Du, Zhongkai Sun
arXiv:2605.03762v2 Announce Type: replace
Abstract: Forecasting is a composite capability encompassing information gathering, evidence integration, situational assessment, and decision-making. Large...
By Yiding Ma, Chengyun Ruan, Kaibo Huang, Jin Tan, Zhongliang Yang, Linna Zhou
arXiv:2609.21190v1 Announce Type: cross
Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...
By George Ma, Benjamin Mikek, Haoyu Li, Ferhat Erata, Yuhao Zhang, Zeren Shui, Behrooz Omidvar Tehrani, Jun Huan, Murali Krishna Ramanathan, Somayeh Sojoudi, Hao Zhou, Anoop Deoras