JIT‑Agent is a model that automatically generates task‑adaptive agent harnesses for any off‑the‑shelf LLM, replacing manual, task‑specific harness design. It learns to compose, repair, and evolve harnesses using a fixed four‑module protocol, and its use boosts performance on benchmarks such as DeepSearchQA and OdysseyBench, outperforming several mature agent runtimes. The approach demonstrates that harness intelligence can be trained, transferred, and compounded independently of model scaling.
arXiv:2609.01437v1 Announce Type: cross Abstract: As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly...
arXiv:2606. 14249v1 Announce Type: new Abstract: AI agent performance depends critically on the runtime harness, comprising the prompts, tools, memory, and control flow that mediate how a model observes, reasons, and acts.
The paper introduces a self‑evolving harness framework where a frozen language‑model agent first solves tasks and then edits its own harness based on run records. Using a 49‑line seed harness, the evolved harness improves average scores on in‑distribution benchmarks by 4.48 points and on out‑of‑distribution benchmarks by 12.64 points, surpassing Codex on the former and matching it on the latter. Continued evolution on a specific out‑of‑distribution benchmark further raises performance, and the study analyzes emergent mechanisms such as output truncation and history compaction.
arXiv:2608. 08466v1 Announce Type: new Abstract: Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the \emph{harness}---is typically treated as a fixed artifact after deployment.
EVOHARNESSBENCH is a new benchmark that tests how LLM-based agents handle changes in their tool, skill, and agent harnesses over time. It includes 17 deterministic harness streams with 802 tasks, 520 tools, 42 skills, and 62 agents, and evaluates agents in two settings: deployment evaluation and self‑evolving adaptation evaluation. The study finds that harness expansion can cause forgetting, adaptation gains are inconsistent, and preserving old competence does not always aid new capability adaptation, highlighting harness evolution as a distinct challenge for agent development.
arXiv:2609.04280v2 Announce Type: replace-cross Abstract: Modern LLM-based agents operate through a harness of tools, reusable skills, and specialist agents that shapes what they observe and what the...
The paper introduces a framework for evaluating large‑language‑model agents by focusing on the interaction between the model and its execution harness. It defines five complementary metrics—Valid Completion Rate, Model‑Harness Fit, Cross‑Model Fit, Interaction Residuals, and Token Load—to assess how reliably agents finish tasks, how well harnesses support specific models, and the overall suitability across models. Using these metrics, the authors evaluate four general‑purpose harnesses and two native systems on six benchmarks, finding that harness rankings differ by model and task and that some harnesses consistently achieve high valid completion rates.
arXiv:2610.15221v1 Announce Type: new Abstract: AI agents can evolve by using environment feedback to revise their harnesses, the software managing model inputs, tool use, memory, and execution, whil...
arXiv:2609.14857v1 Announce Type: new Abstract: Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve executio...
arXiv:2609.24974v1 Announce Type: cross Abstract: Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain...
arXiv:2609.01481v1 Announce Type: new Abstract: This paper studies autonomous software development, in which LLM-based coding agents transform high-level requirements into complete, functional, and u...
arXiv:2608. 06301v1 Announce Type: new Abstract: As LLMs are increasingly deployed within agentic systems, their capabilities depend not only on the model weights but also on the harness: the prompts, tools, control flow, memory, and orchestration code surrounding them.