arXiv Computation and Language

Harness-Zero: Harness Distillation via Agent-as-Harness

arXiv AI
Nov 6

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

The paper investigates whether a language model’s inherent task‑solving ability predicts its ability to self‑evolve its external harness—comprising prompts, skills, memories, and tools—by updating these components based on execution evidence. Two capabilities are examined: harness‑updating (producing useful updates) and harness‑benefit (gaining from those updates). Findings show that harness‑updating performance is largely independent of base capability, while harness‑benefit peaks at mid‑tier models and declines for both weaker and stronger models, suggesting that resources should focus on improving the agent’s core task‑solving skills rather than its evolution mechanism.

By Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi, Yisi Sang, Bing He, Zewen Liu, Tianxin Wei, Zongyu Wu, Zhiwei Zhang, Dakuo Wang, Xiang Zhang, Benoit Dumoulin, Cihang Xie, Yuyin Zhou, Suhang Wang, Hanqing Lu
arXiv Computation and Language
Sep 2

HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

arXiv:2609.01437v1 Announce Type: cross Abstract: As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly...

By Yuhao Wu, Jingyuan Zhang, Jiajun Shi, Xinping Lei, Qingshui Gu, Yuxuan Zhang, Zexuan Wang, Chen He, Chen Huang, Maojia Song, Zhiyuan Zeng, Shaowen Wang, Jinkai Liu, Yunfeng Shi, Jiaheng Liu, Shen Yan, Wenhao Huang, Ge Zhang, Wenxuan Zhang
arXiv AI
Sep 10

Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails

arXiv:2609.09134v1 Announce Type: new Abstract: Agent harnesses (the system prompt, tool set, execution hooks, and context-management scaffolding around a model) are a critical determinant of agentic...

By Zhou Yu, Bin Bi, Shiva Kumar Pentyala, Shubham Mehrotra, Sougata Chaudhuri, Shilpa Bhagavath, Zeyuan Chen, Ran Xu, Phil Mui, James Zhu, Sitaram Asur
arXiv Computation and Language
Sep 7

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

EVOHARNESSBENCH is a new benchmark that tests how LLM-based agents handle changes in their tool, skill, and agent harnesses over time. It includes 17 deterministic harness streams with 802 tasks, 520 tools, 42 skills, and 62 agents, and evaluates agents in two settings: deployment evaluation and self‑evolving adaptation evaluation. The study finds that harness expansion can cause forgetting, adaptation gains are inconsistent, and preserving old competence does not always aid new capability adaptation, highlighting harness evolution as a distinct challenge for agent development.

By Zixuan Ke, Vaidehi Patil, Haizhou Shi, Yang Li, Ye Liu, Sarath Shekkizhar, Anurag Koul, Jiayu Wang, Xuan Phi Nguyen, Semih Yavuz, Mohit Bansal, Shafiq Joty
arXiv Computation and Language
Sep 14

EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness?

arXiv:2609.04280v2 Announce Type: replace-cross Abstract: Modern LLM-based agents operate through a harness of tools, reusable skills, and specialist agents that shapes what they observe and what the...

By Zixuan Ke, Vaidehi Patil, Haizhou Shi, Yang Li, Ye Liu, Sarath Shekkizhar, Anurag Koul, Jiayu Wang, Xuan Phi Nguyen, Semih Yavuz, Mohit Bansal, Shafiq Joty
arXiv Computation and Language
Oct 8

CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolution

arXiv:2610.10426v1 Announce Type: new Abstract: Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Exist...

By Jixuan Chen, Jiaxin Zhang, Qinyuan Ye, Yada Pruksachatkun, Haoxiang Zhang, Jingming Zhuo, Yifan Zhang, Yutong Dai, Juntao Tan, Xiangyu Peng, Silvio Savarese, Zeyuan Chen, Lianhui Qin, Chien-Sheng Wu
arXiv Machine Learning
Sep 22

CHART: A Harness-Rotation Curriculum for Harness-Robust Search Agents

The paper introduces CHART, a curriculum that rotates harnesses during training to teach search agents parallel search strategies robustly across different harness configurations. Unlike static harness augmentation, CHART gradually consolidates behavior by graduating learned harnesses and replacing them, maintaining a reward gap that drives learning. Experiments show CHART enables agents to parallelize on 89% of held‑out harnesses, improves performance on a new QA task by 5.6pp, and benefits more from meta‑harness search than baselines.

By Xinlu Zhang, Ying-Chun Lin, Zhihan Zhang, Besnik Fetahu, Xi Chen
arXiv AI
Oct 5

Harness-Aware Distillation for Small Language Model Agents

The paper introduces Harness-Aware Distillation (HAD), a method for training smaller language model agents that preserves the surrounding harness—software managing context, tools, and feedback—while focusing distillation on the teacher’s contributions beyond the harness. HAD combines an action preference that contrasts teacher actions with and without harness information, and a validity check that filters out contradictory preference pairs. Experiments on long-horizon agent benchmarks show that HAD outperforms standard on‑policy distillation, reducing unproductive loops and improving error recovery without requiring task rewards or future information.

By Moonseok Choi, Taehong Moon, Giung Nam, Juho Lee
arXiv Machine Learning
Aug 27

JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution

JIT‑Agent is a model that automatically generates task‑adaptive agent harnesses for any off‑the‑shelf LLM, replacing manual, task‑specific harness design. It learns to compose, repair, and evolve harnesses using a fixed four‑module protocol, and its use boosts performance on benchmarks such as DeepSearchQA and OdysseyBench, outperforming several mature agent runtimes. The approach demonstrates that harness intelligence can be trained, transferred, and compounded independently of model scaling.

By Guibin Zhang, Leo Lu, Fangzhou Xie, Kang Zhu, Junhao Wang, Zhifei Xie, Zhaochen Yu, Zihang Liu, Zhongxiang Sun, Qiankun Li, Yue Liao, Heng Chang, Xiaobin Hu, Qibing Ren, Wangchunshu Zhou, Shuicheng Yan