arXiv Computation and Language By Michael Nguyen, Wei Chen Tan, Nurul Aisyah Hassan, Arvind Raman, Li Hua Lim, Ahmad Faiz Razak

Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

Read the original on arXiv Computation and Language →

The paper introduces HARNESSEVO, a method that decomposes a large language model’s harness into four independently evolvable components—role, task‑strategy, tool/format‑rules, and reflection/control. Experiments on ALFWorld show that overall success rates are similar to flat‑string evolution, but the reflection/control component alone accounts for most of the performance gains. The study also finds that evenly distributing optimization budget across all slots can be detrimental; concentrating resources on the high‑credit control slot recovers lost performance, while on WebShop all slots remain ineffective, suggesting task‑specific differences in harness value.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Nov 4

Latent Harness: Compiling Agent Behaviors into Post-Block Activation Shifts

The paper introduces Latent Harness, a method that compiles external agent behaviors into a small rank‑8 activation shift applied to every decoder block of a frozen language‑model backbone, affecting only 0.03‑0.05% of parameters. This compiled shift eliminates the need for on‑call harness text, offers runtime controls for injection strength, phase, and mixing, and recovers most of the performance gains of the original harness across multiple settings. The approach also enables zero‑shot transfer across tasks, composability in shift space, and improves budget‑matched harness search, outperforming LoRA with far fewer parameters.

By Ran Li, Shimin Di, Zeyu Ren, Ling Yue, Jiachuan Wang, Lei Chen
arXiv AI
Nov 6

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

The paper investigates whether a language model’s inherent task‑solving ability predicts its ability to self‑evolve its external harness—comprising prompts, skills, memories, and tools—by updating these components based on execution evidence. Two capabilities are examined: harness‑updating (producing useful updates) and harness‑benefit (gaining from those updates). Findings show that harness‑updating performance is largely independent of base capability, while harness‑benefit peaks at mid‑tier models and declines for both weaker and stronger models, suggesting that resources should focus on improving the agent’s core task‑solving skills rather than its evolution mechanism.

By Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi, Yisi Sang, Bing He, Zewen Liu, Tianxin Wei, Zongyu Wu, Zhiwei Zhang, Dakuo Wang, Xiang Zhang, Benoit Dumoulin, Cihang Xie, Yuyin Zhou, Suhang Wang, Hanqing Lu
arXiv AI
Oct 19

Harness-Delta Attribution: Dissecting Gains From Harness Evolution

The paper introduces Harness-Delta Attribution (HDA), a protocol that breaks down the performance gains from harness evolution into overfitting, test-time scaling, and residual generalizable improvement. Applying HDA to 11 executor models and three harness-evolution frameworks across various tasks, the study finds that over half of the search-set gains are due to overfitting or scaling, with limited transfer to held-out tasks. The results highlight the need for attribution-aware evaluation in future harness evolution research.

By Wenxuan Ding, Qin Lu, Changlong Yu, Sha Li, Shuowei Jin, Xin Liu, Zhenyu Shi, Greg Durrett