arXiv Computation and Language

Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

The paper introduces HARNESSEVO, a method that decomposes a large language model’s harness into four independently evolvable components—role, task‑strategy, tool/format‑rules, and reflection/control. Experiments on ALFWorld show that overall success rates are similar to flat‑string evolution, but the reflection/control component alone accounts for most of the performance gains. The study also finds that evenly distributing optimization budget across all slots can be detrimental; concentrating resources on the high‑credit control slot recovers lost performance, while on WebShop all slots remain ineffective, suggesting task‑specific differences in harness value.

arXiv Computation and Language
Nov 4

Latent Harness: Compiling Agent Behaviors into Post-Block Activation Shifts

The paper introduces Latent Harness, a method that compiles external agent behaviors into a small rank‑8 activation shift applied to every decoder block of a frozen language‑model backbone, affecting only 0.03‑0.05% of parameters. This compiled shift eliminates the need for on‑call harness text, offers runtime controls for injection strength, phase, and mixing, and recovers most of the performance gains of the original harness across multiple settings. The approach also enables zero‑shot transfer across tasks, composability in shift space, and improves budget‑matched harness search, outperforming LoRA with far fewer parameters.

By Ran Li, Shimin Di, Zeyu Ren, Ling Yue, Jiachuan Wang, Lei Chen
arXiv AI
Nov 6

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

The paper investigates whether a language model’s inherent task‑solving ability predicts its ability to self‑evolve its external harness—comprising prompts, skills, memories, and tools—by updating these components based on execution evidence. Two capabilities are examined: harness‑updating (producing useful updates) and harness‑benefit (gaining from those updates). Findings show that harness‑updating performance is largely independent of base capability, while harness‑benefit peaks at mid‑tier models and declines for both weaker and stronger models, suggesting that resources should focus on improving the agent’s core task‑solving skills rather than its evolution mechanism.

By Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi, Yisi Sang, Bing He, Zewen Liu, Tianxin Wei, Zongyu Wu, Zhiwei Zhang, Dakuo Wang, Xiang Zhang, Benoit Dumoulin, Cihang Xie, Yuyin Zhou, Suhang Wang, Hanqing Lu
arXiv AI
Oct 19

Harness-Delta Attribution: Dissecting Gains From Harness Evolution

The paper introduces Harness-Delta Attribution (HDA), a protocol that breaks down the performance gains from harness evolution into overfitting, test-time scaling, and residual generalizable improvement. Applying HDA to 11 executor models and three harness-evolution frameworks across various tasks, the study finds that over half of the search-set gains are due to overfitting or scaling, with limited transfer to held-out tasks. The results highlight the need for attribution-aware evaluation in future harness evolution research.

By Wenxuan Ding, Qin Lu, Changlong Yu, Sha Li, Shuowei Jin, Xin Liu, Zhenyu Shi, Greg Durrett
arXiv AI
Sep 10

Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails

arXiv:2609.09134v1 Announce Type: new Abstract: Agent harnesses (the system prompt, tool set, execution hooks, and context-management scaffolding around a model) are a critical determinant of agentic...

By Zhou Yu, Bin Bi, Shiva Kumar Pentyala, Shubham Mehrotra, Sougata Chaudhuri, Shilpa Bhagavath, Zeyuan Chen, Ran Xu, Phil Mui, James Zhu, Sitaram Asur
arXiv AI
Jul 9

The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI

arXiv:2607. 06906v1 Announce Type: new Abstract: Agentic AI development today runs on token maxing: buying capability with tokens -- longer reasoning traces, more turns, wider tool payloads, bigger replayed contexts -- so tokens per task grow faster than task value.

By Muayad Sayed Ali, Aliaksandra Novik, Anji Boddupally, Artem Yavorskyi, Chris Nickerson, Daniel Rica, Emily DuGranrut, Felix Leung, Garrett Prince, Grace Barnett, Heath Robinson, Hosain Al Ahmad, Jesse Resnick, Juan Carlos Farah, Jyothi Swaroop Meruga, Leonid Kuznetsov, Luke Gorham, Marie Schmoll, Michael Paciullo, Saumya Das, Sharath Sheripally, Tommy Griscom, Mykyta Osadchyi, Neha Mantri, Nick Westrum, Olivia Benowitz, Parikshith Kulkarni, Radik Chernyshov, Rakshith Vasudev, Rohith Nadimpally, Vikas Gangadevi, Waseem AlShikh
arXiv AI
Oct 9

Harness Evolution Hits a Ceiling: When Weight Training Should Begin

The paper investigates when to evolve a harness versus training model weights for long‑horizon LLM agents. By labeling failed trajectories with the first signal that fires, the authors separate process failures from content failures, showing that harness evolution fixes the former while weight training addresses the latter. Experiments on DeepPlanning and WebArena‑Lite demonstrate significant performance gains, with LoRA adapters trained on evolved harness trajectories further improving held‑out scores and reducing content failures.

By Yuan Tian, Bing Hu, Hao Wang, Binghang Lu, Fang Wu
arXiv Machine Learning
Sep 22

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

The paper introduces Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), a method that applies regularization principles to the iterative editing of an LLM agent’s harness—prompts, control flow, tooling, memory, and context management. RRSI limits the number of edits per candidate, encourages novel trajectories, and uses a critic and pruner to filter out benchmark‑specific or ineffective changes, thereby favoring reusable agent mechanisms. Experiments on eight benchmarks show RRSI improves performance by up to 14.1 points on the training split and 4.7 points on out‑of‑distribution tests, while reducing policy token usage by 30% compared to unregularized evolution.

By Peng Xia, Rujun Han, Zifeng Wang, Yanfei Chen, Yufan Zhang, Yoonho Lee, Chengsong Huang, Han Yu, Zhongying CuiZhu, Yifei Ming, Huaxiu Yao, Burak Gokturk, Tomas Pfister, Chen-Yu Lee
arXiv AI
3d ago

Multi-Task Self-Evolving Harnesses with the Agent as Its Own Optimizer

The paper introduces a framework for self‑evolving harnesses in which a frozen language‑model agent first solves tasks and then, as a proposer, edits its own harness based on complete run records. The method trains on diverse benchmarks, separating training and held‑out tasks, and evaluates on five unseen out‑of‑distribution benchmarks. Experiments show that harnesses evolved by GLM‑5.3 and GPT‑5.6 Sol achieve performance comparable to or exceeding Claude Code and Codex, with further improvements after continued evolution on Claw‑Eval.

By Qiankai Xu