arXiv AI By Yitong Lin, Jianda Chen, Yihao Hu, Pan Wang, Hang Wang, Zhihao Wen, Xiaoxue Ren, Wei Wu

Verify by Doing: Co-Evolving Agent Execution and Evaluation without External Feedback

Read the original on arXiv AI →

The paper introduces Verify by Doing (VbD), a method that co‑evolves an Actor and an Evaluator within a single task while keeping model weights fixed. The Evaluator actively inspects artifacts and environment state using tools, generating an internal signal that guides both output selection and updates to execution and verification procedures. Experiments on AutomationBench, τ³ Banking, and GDPval show VbD outperforms other methods that lack external feedback, with gains of 3.3, 4.1, and 2.3 percentage points on the main backbone, and ablations confirm the importance of active verification and Evaluator evolution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 19

EvalResearchBench: Can AI Agents Build Their Own Feedback for Recursive Self-Improvement?

EvalResearchBench (ERB) is a benchmark that tests whether AI agents can autonomously design evaluation tasks for recursive self‑improvement. In ERB, a researcher agent selects or synthesizes tasks, builds graders, and refines them through pilot tests before producing a final evaluator that runs within fixed time and API budgets. Experiments with nine researchers and thirteen candidate models show that the best automated evaluators achieve about 75% concordance with target benchmarks, below the 91% ceiling set by target disagreements, and that human‑designed task samples still outperform many automated approaches.

By Yaolun Zhang, Tianyi Xu, Yujie Zhao, Jishen Zhao, Qingyun Wu, Huazheng Wang
arXiv AI
Oct 5

VERSE: Verified Self-Evolving Optimizer for Agent Harnesses

The paper introduces VERSE, a Verified Self‑Evolving optimizer that enhances LLM agent harnesses by allowing the optimizer to test edits, replay failures, and perturb steps while tracking fixes and regressions. VERSE builds its own tools for failure analysis, verification, training audits, and workflow control, and uses this feedback to revise the harness’s prompts, skills, tools, hooks, and notes without changing model weights. In experiments across five executors and multiple languages, VERSE improves all evaluated harness optimizers, achieving higher accuracy on held‑out and out‑of‑distribution tasks compared to the strongest baselines.

By Zekai Wang, Yingqiang Ge, Zekun Wang, Hai Wang, Yuhui Xu, Joshua Frandsen, Shancong Fu, Ashia C. Wilson, Chandan K. Reddy
arXiv AI
Nov 6

Beneath the Surface: What We Overlook When We Talk About Agent Evolution

The paper investigates how agent harness evolution supports recursive self‑improvement without changing model weights, assessing four methods across general and coding tasks. Repeated tests and component interventions show that all evolved agents acquire new tasks while losing some previously mastered ones, with reliability gains sometimes offset by higher deployment costs. The study highlights that the highest‑scoring agent does not always deliver the greatest mean improvement, and that stricter admission criteria can eliminate useful alternatives; it proposes preservation‑aware evaluation and a follow‑up tool, EvoAudit, to refine agent design.

By Xiaobo Wang, Zimu Wang, Siyu Ye, Jiaqi Li, Min Tang, Jun Bai, Yuling Shi, Ruobing Wang, Jingjing Zhang, Chaofan Wang, Kai Wu, Mengnan Qi, Kai Cai, Xiaodong Gu, Qi Liu, Zilong Zheng