RSIHub is an open‑source, composable framework designed to streamline the development, comparison, and extension of agent‑evolution methods. It introduces reusable recipes and operators that manage mutable resources, while providing execution, evaluation, recovery, and lineage tracking. The framework was used to implement and evaluate several methods—AHE, Hyperagents, A‑Evolve, and GEPA—across tasks like MiniSWE, Codex on Terminal Bench 2, and τ³ Banking, achieving up to a 13.5 percentage‑point improvement in overall pass@1 scores.
arXiv:2606. 17546v1 Announce Type: new Abstract: Self-evolving LLM-based agents improve mainly by changing their agent harness: the structured execution layer around a base model, including prompts, memory, tools, middleware, runtime state, and the model-tool interaction loop.
arXiv:2606. 14249v1 Announce Type: new Abstract: AI agent performance depends critically on the runtime harness, comprising the prompts, tools, memory, and control flow that mediate how a model observes, reasons, and acts.
Evo$^2$ is a framework that jointly evolves a language model’s weights and its harness—the set of prompts, tools, and feedback mechanisms—using a dual-role pipeline where a single policy acts as both a Proposer and a Reflector. The training pipeline EvoTIDE combines supervised fine-tuning with interleaved GRPO to refine both roles based on harness edits and execution outcomes. Experiments on coding benchmarks show Evo$^2$ outperforms existing self‑evolution methods, achieving higher scores on Polyglot and SWE‑bench Verified, and its evolved harnesses transfer across tasks and models.
arXiv:2610.18813v1 Announce Type: new Abstract: Harness self-evolution revises an agent's software scaffolding, including prompts, memory, workflows, and executable components, while keeping model we...
The paper introduces ARTEMIS, a no-code evolutionary optimization platform that automatically tunes large language model (LLM) agents by jointly optimizing prompts, tool descriptions, and parameters using semantically-aware genetic operators. Starting from a benchmark script and natural language goals, ARTEMIS discovers configurable components, extracts performance signals from execution logs, and evolves configurations without architectural changes. Experiments on four agent systems show significant gains: a 13.6% increase in acceptance rate for the ALE Agent, a 10.1% performance boost for the Mini‑SWE Agent, a 36.9% token‑reduction for the CrewAI Agent, and a 22% accuracy improvement for the MathTales‑Teacher Agent using a smaller open‑source model.
The paper investigates how agent harness evolution supports recursive self‑improvement without changing model weights, assessing four methods across general and coding tasks. Repeated tests and component interventions show that all evolved agents acquire new tasks while losing some previously mastered ones, with reliability gains sometimes offset by higher deployment costs. The study highlights that the highest‑scoring agent does not always deliver the greatest mean improvement, and that stricter admission criteria can eliminate useful alternatives; it proposes preservation‑aware evaluation and a follow‑up tool, EvoAudit, to refine agent design.
arXiv:2610.15765v1 Announce Type: cross Abstract: Large language model agents are emerging as an important new class of users of desktop applications. However, existing desktop interfaces were design...
arXiv:2611.07595v1 Announce Type: cross Abstract: Agent exploration produces valuable procedural knowledge, yet much of this experience remains confined to transient execution traces and is not syste...
StarHarness is a framework that evolves environment‑specific agent harnesses while keeping model weights fixed. It builds a compact evolution pool by stratifying tasks based on baseline failure behavior, separating proposer‑visible search tasks from hidden selection tasks, and reserving held‑out tasks for generalization evaluation. In enterprise benchmarks such as ITBench SRE, EnterpriseOps‑Gym ITSM, and AutomationBench Finance, harness evolution boosts full‑benchmark performance by 20‑35 percentage points after 4‑12 accepted changes per environment, with gains that persist on unseen tasks and transfer across GPT and Qwen model families.
The paper introduces Agent S4, an autonomous system that learns to design and refine user interfaces for computer‑use agents by observing task execution without explicit success labels. By evolving a repository of interfaces through population‑based selection on unseen validation tasks, Agent S4 achieves state‑of‑the‑art performance on OSWorld V2 and significant gains on ScienceBoard, outperforming large commercial models at lower cost. The work demonstrates that interface design can be treated as an agent capability, enabling continuous improvement beyond fixed, hand‑crafted interfaces.
The paper introduces the Self‑Evolving Agent Harness, a framework that allows an agent’s software scaffolding—prompts, memory, workflows, and executable components—to evolve while keeping the underlying model weights unchanged. It formalizes a feedback‑driven process where accepted revisions persist across runs, separating update targets from proposal and admission mechanisms, and distinguishes changes to interpreted state from revisions to tools and code. The authors analyze evolutionary search, provide generalization and retention bounds under PAC‑learning assumptions, and discuss the impact of persistent harness revisions on knowledge sharing, organizational workflows, and broader implications for scientific reliability, safety, and accountability.
Harbor Adapters is a unified evaluation infrastructure that ports over 80 agentic benchmarks, enabling arbitrary agents to be tested across complex environments. The authors performed a large‑scale evaluation of 8 models on 54 benchmarks, using Terminus‑2 and three native harnesses, revealing detailed agent capabilities and failure modes. They also created Harbor‑Index, a curated set of 82 challenging tasks from 29 benchmarks, designed to be affordable yet comprehensive, with the best model achieving a 28.0% pass rate.