arXiv:2607. 13034v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires.
The paper proposes a claim‑specific verification audit for modular agents that replaces aggregate task scores with evidence‑based evaluations. Each agent conclusion is recorded with supporting evidence and classified as supported, unsupported, unresolved, or not evaluated, along with the boundary of validity. The audit employs three tools—oracle policies, perfect component replacements, and verifier‑score tests—to trace value changes, locate lost value, and assess verifier effectiveness, demonstrated on a portfolio‑allocation agent in a synthetic market.
The paper introduces agentic meta‑reasoning, a structured inference‑time framework that explicitly manages control decisions—such as selecting partial work, restarting, or stopping—during long‑horizon agentic tasks. By delegating task execution to workers and consolidating decisions through a lightweight controller that references persistent memory, the method reduces the need to replay full histories. Experiments on ProgramBench and other benchmarks show that meta‑reasoning improves performance over direct control baselines, especially as computation budgets increase, and reveals greater reuse of earlier work and higher solution coverage.
The paper investigates how multiple pre‑action controls—authority, resource, and evidence gates—interact in agentic AI systems. It formalizes remediation‑induced control coupling, showing that remediation can invalidate earlier judgments and that the order of remediation matters. The authors propose a remediate‑and‑regate protocol to restore soundness, analyze non‑commuting remediation operators, and demonstrate the approach on a deterministic open‑data artifact with three published engines.
Two prompts can request the same code change and produce the same correct patch, yet cause a coding agent to perform radically different kinds and amounts of work. We study this effect in a preregistered benchmark spanning 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real agent harnesses.
arXiv:2607. 04528v1 Announce Type: new Abstract: Software-agent benchmarks usually report whether an agent solves a task, but the agent reaches that outcome through a harness that controls what it sees, which actions it can take, which failures are repaired, which states are verified, and which evidence is logged.
arXiv:2608.28795v1 Announce Type: cross Abstract: Modern artificial-intelligence coding agents can be equipped with tools for checking their own work e.g. a linter, a boot probe, a shell, a screensho...
The paper introduces CABRA, a synthetic benchmark that constructs coding tasks from scratch using call‑graph transformations and scales difficulty along four axes: function traversal, search, runtime resolution, and instruction following. Experiments with eight large language models and six coding agents on 6,840 CABRA tasks show that while LLM accuracy degrades with task size, agents maintain high performance by leveraging tools; larger tasks trigger more tool calls, and a separate study on SWE‑bench verifies that tool‑call counts better predict agent accuracy than lines of code edited. The authors argue that CABRA exposes weaknesses in LLMs that are hidden by tool use and highlights the importance of code‑understanding abilities beyond mere editing. whyItMatters":"The study demonstrates that coding agents’ performance is more dependent on code‑understanding than on the amount of code edited, revealing a critical bottleneck for current coding agents."
arXiv:2607. 15854v1 Announce Type: cross Abstract: Coding agents can fix a failing example without preserving the domain rule that made it fail, so later generations can repeat the same plausible mistake.
The paper discusses how the same computational task can require different reasoning strategies depending on the order in which evidence arrives, introducing the concept of an "information‑slack dilemma." It argues that early computation may be useful only if its benefits outweigh the costs of later verification, invalidation, and recovery, and proposes a research agenda focused on selective recovery and predictive policies. The authors emphasize evaluating these approaches by separating early‑execution effects, deployment value versus full‑input alternatives, and the added value of predictive policies while considering shared‑resource costs.
arXiv:2606. 14357v1 Announce Type: cross Abstract: Frontier coding models may spend substantial capacity learning not only program behavior, but also accidental entropy in human repositories.
arXiv:2607. 17352v1 Announce Type: new Abstract: Designing effective Lean proof agents is a central challenge in formal mathematical reasoning.