arXiv:2608. 08466v1 Announce Type: new Abstract: Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the \emph{harness}---is typically treated as a fixed artifact after deployment.
By Tailin Zhou
The paper introduces Latent Harness, a method that compiles external agent behaviors into a small rank‑8 activation shift applied to every decoder block of a frozen language‑model backbone, affecting only 0.03‑0.05% of parameters. This compiled shift eliminates the need for on‑call harness text, offers runtime controls for injection strength, phase, and mixing, and recovers most of the performance gains of the original harness across multiple settings. The approach also enables zero‑shot transfer across tasks, composability in shift space, and improves budget‑matched harness search, outperforming LoRA with far fewer parameters.
By Ran Li, Shimin Di, Zeyu Ren, Ling Yue, Jiachuan Wang, Lei Chen
The paper investigates whether a language model’s inherent task‑solving ability predicts its ability to self‑evolve its external harness—comprising prompts, skills, memories, and tools—by updating these components based on execution evidence. Two capabilities are examined: harness‑updating (producing useful updates) and harness‑benefit (gaining from those updates). Findings show that harness‑updating performance is largely independent of base capability, while harness‑benefit peaks at mid‑tier models and declines for both weaker and stronger models, suggesting that resources should focus on improving the agent’s core task‑solving skills rather than its evolution mechanism.
By Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi, Yisi Sang, Bing He, Zewen Liu, Tianxin Wei, Zongyu Wu, Zhiwei Zhang, Dakuo Wang, Xiang Zhang, Benoit Dumoulin, Cihang Xie, Yuyin Zhou, Suhang Wang, Hanqing Lu
Agentic AI development today runs on token maxing: buying capability with tokens -- longer reasoning traces, more turns, wider tool payloads, bigger replayed contexts -- so tokens per task grow faster than task value. Falling per-token prices mask the pattern; total spend rises anyway.
The paper introduces Harness-Delta Attribution (HDA), a protocol that breaks down the performance gains from harness evolution into overfitting, test-time scaling, and residual generalizable improvement. Applying HDA to 11 executor models and three harness-evolution frameworks across various tasks, the study finds that over half of the search-set gains are due to overfitting or scaling, with limited transfer to held-out tasks. The results highlight the need for attribution-aware evaluation in future harness evolution research.
By Wenxuan Ding, Qin Lu, Changlong Yu, Sha Li, Shuowei Jin, Xin Liu, Zhenyu Shi, Greg Durrett
arXiv:2608. 09629v1 Announce Type: new Abstract: Self-evolving agents are usually built around prescribed optimization pipelines: the framework decides how to gather evidence, revise a persistent artifact, select candidates, and stop.
By Hui Xue, Fan Yang