StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments

StarHarness is a framework that evolves environment‑specific agent harnesses while keeping model weights fixed. It builds a compact evolution pool by stratifying tasks based on baseline failure behavior, separating proposer‑visible search tasks from hidden selection tasks, and reserving held‑out tasks for generalization evaluation. In enterprise benchmarks such as ITBench SRE, EnterpriseOps‑Gym ITSM, and AutomationBench Finance, harness evolution boosts full‑benchmark performance by 20‑35 percentage points after 4‑12 accepted changes per environment, with gains that persist on unseen tasks and transfer across GPT and Qwen model families.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents
• benchmarks