arXiv:2608. 20169v1 Announce Type: cross Abstract: We present a novel approach to efficient LLM agent harness optimization through adaptive validation task selection.
arXiv:2608. 06301v1 Announce Type: new Abstract: As LLMs are increasingly deployed within agentic systems, their capabilities depend not only on the model weights but also on the harness: the prompts, tools, control flow, memory, and orchestration code surrounding them.
arXiv:2607. 12227v1 Announce Type: new Abstract: We revisit the evaluation of automatic harness evolution for LLM agents.
The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.
Large Language Models (LLMs) have driven rapid progress in autonomous agents, yet standard evaluations remain confined to static task solving. An emerging frontier is harness evolution---the agent's capacity to autonomously optimize its own operating harness.
arXiv:2609.14857v1 Announce Type: new Abstract: Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve executio...
arXiv:2607. 26784v1 Announce Type: new Abstract: Large language model agents often encounter related yet distinct tasks that share reusable solution patterns.
The paper introduces HarnessLens, a budget‑aware framework that evolves language‑model agent harnesses by jointly exploring task spaces and user‑configurable components. It derives candidate modifications from execution trajectories and selectively verifies them on behavior‑relevant tasks using an attributable‑evidence gate, thereby avoiding wasteful rollouts on unrelated behaviors. Experiments on three harnesses and four benchmarks show that HarnessLens improves held‑out performance by 7.6‑13.6% while using less evaluation budget than existing methods.
arXiv:2608. 09168v1 Announce Type: new Abstract: Agent skills are increasingly used to equip large language model (LLM) agents with reusable procedural knowledge.
arXiv:2605. 28556v2 Announce Type: replace Abstract: As agent capabilities advance, existing benchmarks, such as $\tau^2$-Bench, are becoming increasingly saturated.
arXiv:2606. 21140v2 Announce Type: replace-cross Abstract: Rapid advances in large language models have improved the task-solving capabilities of command-line-interface (CLI)-based agents, whose CLIs determine how models invoke tools, maintain interaction history, and recover from failures.
arXiv:2609.01437v1 Announce Type: cross Abstract: As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly...
The study investigates whether large language models (LLMs) can automatically improve the harnesses used for hardware design verification. By evolving harnesses around a fixed subject model on 12 proprietary root‑cause localization tasks, the researchers found that automatically evolved harnesses increased completed attempts by 71‑76% and task coverage by 80‑100%, though overall correct attempts improved only 18‑24%. Despite these gains, the evolved harnesses did not consistently consolidate into a single dominant solution across tasks and metrics, and a separate cross‑benchmark case showed that a repair harness could yield a 35.6% increase in functional passes over a baseline.