arXiv AI By Qianshu Cai, Yonggang Zhang, Jun Nie, Maohao Ran, Huajiang Zheng, Jun Song, Xinmei Tian, Yike Guo, Wei Xue

Safe Harness Self-Evolution: A Theoretical Analysis of Feasibility and Limits

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

Hugging Face Trending Papers
Oct 13

Self-Evolving Agent Harness

The paper introduces the Self‑Evolving Agent Harness, a framework that allows an agent’s software scaffolding—prompts, memory, workflows, and executable components—to evolve while keeping the underlying model weights unchanged. It formalizes a feedback‑driven process where accepted revisions persist across runs, separating update targets from proposal and admission mechanisms, and distinguishes changes to interpreted state from revisions to tools and code. The authors analyze evolutionary search, provide generalization and retention bounds under PAC‑learning assumptions, and discuss the impact of persistent harness revisions on knowledge sharing, organizational workflows, and broader implications for scientific reliability, safety, and accountability.

arXiv AI
Oct 15

Self-Evolving Agent Harness

arXiv:2610.18813v1 Announce Type: new Abstract: Harness self-evolution revises an agent's software scaffolding, including prompts, memory, workflows, and executable components, while keeping model we...

By Chuyang Wei, Haoxiang Guan, Minghang Zhu, Kexin Zhou, Jiyan He, Xiren Zhou, Huanhuan Chen, Yifei Shen
arXiv AI
Sep 17

Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts

arXiv:2609. 18366v1 Announce Type: new Abstract: Reliable agent evaluation is complicated by automatic harness optimization, which repeatedly uses a released benchmark $B_{\mathrm{rel}}$ to guide a Proposer that edits prompts, memory, retrieval, tools, and control code around a fixed target agent.

By Guojun Zhu, Xunheng Huang, Peng Yin, Jiahui Xie, Sanguo Zhang, Doudou Zhou
arXiv AI
Oct 1

Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.

By Qiankai Xu
arXiv AI
Oct 28

When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

The paper introduces RuHarness, a framework that co‑evolves rubrics and harnesses to enable continual self‑improvement of autonomous agents that rely on large language models. RuHarness automatically discovers and reassesses behavioral criteria from execution trajectories, prioritizes unresolved weaknesses, and uses these insights to guide targeted harness updates while keeping the task objective unchanged. Experiments across varied agent tasks and backbone models demonstrate that RuHarness outperforms existing harness‑evolution baselines, especially on unseen held‑out tasks, and that its adaptive rubric prioritization supports sustained improvement and stable generalization over successive evolution rounds.

By Hao Chen, Yukun Yan, Ziyu Han, Ye He, Chuanzhe Guo, Shangzhan Li, Qi Shi, Qingfu Zhu, Maosong Sun, Wanxiang Che