The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2608. 05519v1 Announce Type: new Abstract: Agent benchmarks usually measure task completion and treat resource use as an auxiliary statistic.
The paper investigates how test‑time computation can enhance language models and at what cost, introducing the SELF‑POT benchmark to evaluate this across competition mathematics, competitive programming, and agentic workflows. SELF‑POT separates candidate coverage from final accuracy, tracks correctness transitions under revision, and measures protocol completion alongside task success. Using a unified budget rule, the study compares direct inference, parallel sampling, and self‑revision across five low‑cost reasoning models, revealing that selection rules and failure handling significantly influence gains and cost savings.
arXiv:2606. 04402v1 Announce Type: new Abstract: Modern reasoning models can allocate different amounts of test-time computation, such as thinking tokens, model calls, or compute budget, to different tasks.
The paper introduces agentic meta‑reasoning, a structured inference‑time framework that explicitly manages control decisions—such as selecting partial work, restarting, or stopping—during long‑horizon agentic tasks. By delegating task execution to workers and consolidating decisions through a lightweight controller that references persistent memory, the method reduces the need to replay full histories. Experiments on ProgramBench and other benchmarks show that meta‑reasoning improves performance over direct control baselines, especially as computation budgets increase, and reveals greater reuse of earlier work and higher solution coverage.
The paper investigates how users should set the reasoning‑effort level in large language models, a parameter that trades off inference cost for potential performance gains. Experiments on proprietary and open‑weight models across four benchmarks show that higher effort does not consistently improve accuracy, can be up to 42× more expensive on easy tasks, and alters agent behavior by increasing verification and search steps. The authors propose a simple escalation strategy—starting at the lowest effort and only increasing after a verified failure—that matches or outperforms fixed high‑effort settings while reducing cost.
The paper introduces PORA, a method that teaches large language model agents to decide when to allocate more reasoning steps versus fewer, based on the expected downstream benefit. PORA trains by running each state twice—once with short reasoning and once with long reasoning—and learns to predict the value of extended reasoning. During inference, a controller uses this prediction and remaining budget information to choose the appropriate reasoning length without running both options, improving performance across diverse tasks.