arXiv:2610.10426v1 Announce Type: new
Abstract: Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Exist...
By Jixuan Chen, Jiaxin Zhang, Qinyuan Ye, Yada Pruksachatkun, Haoxiang Zhang, Jingming Zhuo, Yifan Zhang, Yutong Dai, Juntao Tan, Xiangyu Peng, Silvio Savarese, Zeyuan Chen, Lianhui Qin, Chien-Sheng Wu
arXiv:2609.24974v1 Announce Type: cross
Abstract: Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain...
By Haoran Ye, Yuxing Lu, Haonan Dong, Zhaochen Su, Guojie Song
Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Existing harness-model co-evolution approaches improve...
arXiv:2609.39304v1 Announce Type: cross
Abstract: When an off-the-shelf coding agent is used directly as a robot policy, observing a browser-based 3D interface through screenshots and acting by posin...
By Zhijie Wei, Ferris Tan, Jinghui Wang
The paper introduces a framework for self‑evolving harnesses in which a frozen language‑model agent first solves tasks and then, as a proposer, edits its own harness based on complete run records. The method trains on diverse benchmarks, separating training and held‑out tasks, and evaluates on five unseen out‑of‑distribution benchmarks. Experiments show that harnesses evolved by GLM‑5.3 and GPT‑5.6 Sol achieve performance comparable to or exceeding Claude Code and Codex, with further improvements after continued evolution on Claw‑Eval.
By Qiankai Xu
The paper introduces WHALE, a method that alternates between updating a language model’s weights and searching for a better harness (the code that manages context and control flow). By iteratively fine‑tuning the model under the current harness and then optimizing the harness under the updated model, WHALE improves performance across search QA, math reasoning, and chess puzzles, outperforming weight‑only, harness‑only, and Fast‑Slow Training by 4.15–24.38 percentage points in mean@8 accuracy. The approach uses either fixed phase lengths or an adaptive patience rule to decide when to switch phases, and the authors provide code on GitHub.
By Haechan Kim, Yoonho Lee, Gisang Lee, Chelsea Finn, Kangwook Lee
The paper investigates when to evolve a harness versus training model weights for long‑horizon LLM agents. By labeling failed trajectories with the first signal that fires, the authors separate process failures from content failures, showing that harness evolution fixes the former while weight training addresses the latter. Experiments on DeepPlanning and WebArena‑Lite demonstrate significant performance gains, with LoRA adapters trained on evolved harness trajectories further improving held‑out scores and reducing content failures.
By Yuan Tian, Bing Hu, Hao Wang, Binghang Lu, Fang Wu
ActiveSaddler introduces automated curriculum learning for harness optimization, treating the evolving training curriculum as a non‑stationary bandit problem. It identifies reusable failure patterns, estimates learning progress for each, and balances revisiting known weaknesses with exploring new scenarios, allowing the curriculum to co‑evolve with the harness. Experiments on GAIA2 and Terminal‑Bench 2.0 show consistent improvements in harness performance, with Pass@1 gains of 4.4 and 7.5 percentage points over fixed‑order baselines.
By Sungho Park, Wonjoong Kim, Jue Zhang, Wook-Shin Han, Pengfei Gao, Chanyoung Park, Yongqiang Yao, Rao Fu, Elsie Nallipogu, Qingwei Lin, Victor R\"uhle
The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.
By Qiankai Xu
The paper investigates whether a language model’s inherent task‑solving ability predicts its ability to self‑evolve its external harness—comprising prompts, skills, memories, and tools—by updating these components based on execution evidence. Two capabilities are examined: harness‑updating (producing useful updates) and harness‑benefit (gaining from those updates). Findings show that harness‑updating performance is largely independent of base capability, while harness‑benefit peaks at mid‑tier models and declines for both weaker and stronger models, suggesting that resources should focus on improving the agent’s core task‑solving skills rather than its evolution mechanism.
By Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi, Yisi Sang, Bing He, Zewen Liu, Tianxin Wei, Zongyu Wu, Zhiwei Zhang, Dakuo Wang, Xiang Zhang, Benoit Dumoulin, Cihang Xie, Yuyin Zhou, Suhang Wang, Hanqing Lu
EVOHARNESSBENCH is a new benchmark that tests how LLM-based agents handle changes in their tool, skill, and agent harnesses over time. It includes 17 deterministic harness streams with 802 tasks, 520 tools, 42 skills, and 62 agents, and evaluates agents in two settings: deployment evaluation and self‑evolving adaptation evaluation. The study finds that harness expansion can cause forgetting, adaptation gains are inconsistent, and preserving old competence does not always aid new capability adaptation, highlighting harness evolution as a distinct challenge for agent development.
By Zixuan Ke, Vaidehi Patil, Haizhou Shi, Yang Li, Ye Liu, Sarath Shekkizhar, Anurag Koul, Jiayu Wang, Xuan Phi Nguyen, Semih Yavuz, Mohit Bansal, Shafiq Joty
arXiv:2609.04280v2 Announce Type: replace-cross
Abstract: Modern LLM-based agents operate through a harness of tools, reusable skills, and specialist agents that shapes what they observe and what the...
By Zixuan Ke, Vaidehi Patil, Haizhou Shi, Yang Li, Ye Liu, Sarath Shekkizhar, Anurag Koul, Jiayu Wang, Xuan Phi Nguyen, Semih Yavuz, Mohit Bansal, Shafiq Joty