arXiv AI
Jul 15
Rethinking the Evaluation of Harness Evolution for Agents
arXiv:2607. 12227v1 Announce Type: new Abstract: We revisit the evaluation of automatic harness evolution for LLM agents.
HarnessPro: Recursive Harness Self-Improvement with Pareto Optimization for Performance-Cost Trade-offs
Related stories
arXiv AI
Aug 7
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
arXiv:2608. 06301v1 Announce Type: new Abstract: As LLMs are increasingly deployed within agentic systems, their capabilities depend not only on the model weights but also on the harness: the prompts, tools, control flow, memory, and orchestration code surrounding them.
arXiv AI
Oct 1
Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer
The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.
arXiv AI
Oct 15
Harness^2: Recursive Agent Harnessing for an Open World
arXiv:2610.14224v1 Announce Type: new Abstract: LLM agents' capabilities depend not only on their underlying models but also on their execution harnesses. Recursive self-improvement (RSI) offers a pr...
arXiv Computation and Language
Sep 2
HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?
arXiv:2609.01437v1 Announce Type: cross Abstract: As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly...
arXiv Computation and Language
Sep 15
ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
arXiv:2609.14857v1 Announce Type: new Abstract: Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve executio...
arXiv Machine Learning
Sep 22
RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
The paper introduces Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), a method that applies regularization principles to the iterative editing of an LLM agent’s harness—prompts, control flow, tooling, memory, and context management. RRSI limits the number of edits per candidate, encourages novel trajectories, and uses a critic and pruner to filter out benchmark‑specific or ineffective changes, thereby favoring reusable agent mechanisms. Experiments on eight benchmarks show RRSI improves performance by up to 14.1 points on the training split and 4.7 points on out‑of‑distribution tests, while reducing policy token usage by 30% compared to unregularized evolution.
arXiv AI
Oct 21
VibeTraining-Bench: Data-Trainer-Harness Co-Design for Iterative System Improvement
arXiv:2610.26342v2 Announce Type: replace Abstract: Recursive self-improvement (RSI) requires agents to diagnose system bottlenecks and coordinate improvements across training and inference. We refer...
arXiv AI
Sep 30
Mixture of Self-Improving Branches For Agent Harness Optimization
arXiv:2609.37834v1 Announce Type: new Abstract: Harness optimization provides a practical setting for recursive self-improvement (RSI), where agent-generated modifications inform subsequent changes t...
arXiv AI
Sep 30
SelfSearch: Reward-Free Search for Self-Improving Agents
arXiv:2609. 37968v1 Announce Type: new Abstract: Advances in the coding capabilities of LLM agents allow them to inspect and modify their own instructions, tools, and execution procedures.
arXiv AI
Sep 18
SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness
arXiv:2609. 20519v1 Announce Type: new Abstract: As coding agents move from supervised code completion to unattended, around-the-clock exploration, their work expands from isolated predictions into long trajectories of reasoning, tool use, and feedback.
arXiv AI
Aug 11
Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses
arXiv:2608. 08466v1 Announce Type: new Abstract: Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the \emph{harness}---is typically treated as a fixed artifact after deployment.