arXiv AI August 28, 2026 By Jinghan Xu, Yikai Zhang, Aili Chen, Weiyuan Li, Jiaqing Liang, Deqing Yang

Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification

The paper introduces HarnessLens, a budget‑aware framework that evolves language‑model agent harnesses by jointly exploring task spaces and user‑configurable components. It derives candidate modifications from execution trajectories and selectively verifies them on behavior‑relevant tasks using an attributable‑evidence gate, thereby avoiding wasteful rollouts on unrelated behaviors. Experiments on three harnesses and four benchmarks show that HarnessLens improves held‑out performance by 7.6‑13.6% while using less evaluation budget than existing methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Aug 10

Evo-Bench: Can Language Models Improve Agent Harness?

Large Language Models (LLMs) have driven rapid progress in autonomous agents, yet standard evaluations remain confined to static task solving. An emerging frontier is harness evolution---the agent's capacity to autonomously optimize its own operating harness.

llms agents benchmarks

arXiv Machine Learning

Aug 4

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses

arXiv:2608. 01918v1 Announce Type: new Abstract: Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments.

By Luan Zhang, Ruochen Zhou, Dandan Song, Zhengyu Chen, Yuhang Tian, Jun Yang, Huipeng Ma, Chenhao Li, Guangyuan Feng, Xudong Li, Yizhou Jin, Yan Xu

llms agents

arXiv AI

Oct 1

Composing Task-specific Agent Harnesses at Test Time with Reusable Primitives

The paper introduces Harness Primitives—reusable agent harness mechanisms mined from failed task trajectories—and a framework called STITCH that selects and composes these primitives into task‑specific harnesses at test time. This approach avoids generating or debugging harness code for each task, achieving up to 12‑point gains in task success over fixed harness baselines and outperforming human‑designed harnesses like Codex CLI. STITCH also demonstrates minimal test‑time overhead (2.7%) and scales efficiently with the size of the primitive library.

By Peng Kuang, Haibo Jin, Dehao Wu, Feiyang Deng, Xiaopeng Yuan, Jerry Wang, Haohan Wang

llms agents

Hugging Face Trending Papers

Aug 24

ADE: Agentic Data Evolution Framework for Human-Centered Objectives

Aligning large language models to human-centered objectives is difficult when targets are non-executable and context-dependent, limiting reliable verification and scalable supervision. Although synthe...

llms agents benchmarks

arXiv AI

Jun 15

HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry

arXiv:2606. 14249v1 Announce Type: new Abstract: AI agent performance depends critically on the runtime harness, comprising the prompts, tools, memory, and control flow that mediate how a model observes, reasons, and acts.

By Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, Yuan Xie, Heng Qu, Kun Shao, Jian Luan

agents reinforcement-learning benchmarks

arXiv Computation and Language

Oct 8

CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolution

arXiv:2610.10426v1 Announce Type: new Abstract: Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Exist...

By Jixuan Chen, Jiaxin Zhang, Qinyuan Ye, Yada Pruksachatkun, Haoxiang Zhang, Jingming Zhuo, Yifan Zhang, Yutong Dai, Juntao Tan, Xiangyu Peng, Silvio Savarese, Zeyuan Chen, Lianhui Qin, Chien-Sheng Wu

agents reinforcement-learning fine-tuning