arXiv AI
Jul 21
Self-Modifying Lean Proof Agents with Verifier-Grounded Benchmark Coevolution
arXiv:2607. 17352v1 Announce Type: new Abstract: Designing effective Lean proof agents is a central challenge in formal mathematical reasoning.
VeriEvolve: Evaluator-First Evolution for Tool-Using Agents
VeriEvolve is an evaluator-first evolution framework that first builds an executable evaluator from task instructions, then uses this evaluator as a frozen fitness signal while evolving agents. The system generates challenge states and counterfactual test cases to refine the evaluator across rounds, producing 30 evaluators for 30 MCPMark Filesystem tasks with perfect agreement on generated suites and high agreement on held-out states. Evaluator-guided evolution significantly improves externally verified success rates for GLM-5.2 and Claude, and cross-environment studies reveal complementary limits from observability, state representation, and search/execution capacity.
Related stories
arXiv AI
Oct 5
VERSE: Verified Self-Evolving Optimizer for Agent Harnesses
The paper introduces VERSE, a Verified Self‑Evolving optimizer that enhances LLM agent harnesses by allowing the optimizer to test edits, replay failures, and perturb steps while tracking fixes and regressions. VERSE builds its own tools for failure analysis, verification, training audits, and workflow control, and uses this feedback to revise the harness’s prompts, skills, tools, hooks, and notes without changing model weights. In experiments across five executors and multiple languages, VERSE improves all evaluated harness optimizers, achieving higher accuracy on held‑out and out‑of‑distribution tasks compared to the strongest baselines.
arXiv AI
Aug 11
Argus: A General-Purpose Agentic Reasoning Runtime for Long-Horizon Tasks
arXiv:2608. 05144v2 Announce Type: replace Abstract: Long-horizon reasoning requires an agentic runtime that can persist when evidence supports its current approach and pivot when measurements reveal failure, hidden constraints, or a misspecified objective.
arXiv AI
Aug 6
Argus: A General-Purpose Agentic Runtime for Long-Horizon Reasoning
arXiv:2608. 05144v1 Announce Type: new Abstract: Long-horizon reasoning requires an agentic runtime that can persist when evidence supports its current approach and pivot when measurements reveal failure, hidden constraints, or a misspecified objective.
arXiv AI
Oct 15
Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress
arXiv:2610.14146v1 Announce Type: new Abstract: Long-horizon terminal benchmarks measure whether an agent can sustain progress through extended workflows with process rewards, not only whether its fi...
arXiv AI
Jun 26
The Red Queen G\"odel Machine: Co-Evolving Agents and Their Evaluators
arXiv:2606. 26294v1 Announce Type: cross Abstract: Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains.
arXiv Machine Learning
Jul 10
TTHE: Test-Time Harness Evolution
arXiv:2607. 08124v1 Announce Type: cross Abstract: The behavior of an LLM agent is determined not only by the underlying model, but also by its harness: the executable program that constructs context, invokes tools, verifies intermediate results, and recovers from failures.
arXiv AI
Oct 15
Verifier-Harness Co-Evolution for Training-Free Self-Improvement on Open-Ended Tasks
arXiv:2610.17609v1 Announce Type: new Abstract: Training-free self-improvement lets an agent use feedback to revise the response it is about to return and to update the harness state that shapes how...
arXiv AI
Oct 9
Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents
The paper introduces a method for co-evolving inspectable verifiers alongside self-improving agents. By synthesizing verifiers from clustered failures and selecting them based on agreement with a reference set, the authors demonstrate improved held‑out agreement on MBPP+ and outperform a bare LLM judge. They also show that removing anchor guards collapses the verifier into a vacuous grader, yet the collapsed verifier still trains skills effectively, indicating that downstream task scores cannot certify a self‑evolved verifier.
arXiv AI
Oct 15
Davinci-Tool: Scaling Agentic Tool-Use Supervision with Verifiable Executable Environments
arXiv:2610.16867v1 Announce Type: cross Abstract: Reliable agentic tool use requires training data that pair diverse multi-step workflows with executable evidence that the requested outcomes were ach...
arXiv AI
Jul 14
AgentAbstain: Do LLM Agents Know When Not to Act?
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.
arXiv Machine Learning
Oct 7
EvoHarness-RL: Learning Runtime Harness Coordination for Self-Evolving Agents
arXiv:2608.05446v2 Announce Type: replace Abstract: Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, recover from failures, and reuse experie...