Agent Harness Awareness addresses the gap between tool descriptions and real‑time operational suitability by having the host provide a verified, up‑to‑date state snapshot before an agent dispatches an action. In controlled benchmarks, this approach cut observable policy regret by nearly 97 % and boosted single‑dispatch completion from 69.4 % to 95.8 %. In live, fault‑injected tests, verified state eliminated excess operational cost and achieved 100 % completion across all model‑by‑fault scenarios.
arXiv:2608.27808v1 Announce Type: cross Abstract: Self-report is the cheapest oversight channel a deployer has, and on capable computer-use agents (CUAs) it fails precisely where oversight matters. O...
The paper introduces a coroutine-bridge harness that lets a language model emit a Python program to manage tool calls in the CAR-bench evaluation. By decoupling model invocations from tool round-trips, the approach reduces model calls to a median of two per task while maintaining seven agent turns, achieving a median latency of 1.8 s on a Cerebras gpt‑oss‑120b. The harness achieved 60.0 % Pass³ on the official hidden evaluation, outperforming the baseline by 4.5× and matching frontier-model agents on GPT‑5.5, all while keeping the prompt largely cached and minimizing input compute.
The paper introduces prefix executability, a trajectory-level evaluation protocol for tool-using agents that checks whether every prefix of a predicted tool-call sequence remains executable under tool schemas, state preconditions, and data dependencies. It defines metrics such as Prefix Executability Curves, First-Error Depth, Trajectory Executability Rate, horizon-normalized AUC-PEC, and Recovery Rate to quantify reliability. Experiments on BFCL-v3, ToolSandbox, and $ au^3$-bench show that agents can achieve high final success while having low trajectory executability, revealing failure modes that final success metrics miss, and demonstrate that explicit pre-execution plan checking can improve prefix-level reliability without sacrificing terminal success.
HarnessMM (Harness Minus Minus) is a method that improves harness token efficiency by using guarded decision reuse and recursive self‑improvement (RSI). It refines harness configurations from training feedback, reuses resolved decisions to replace later reasoning callbacks, and stores actions in Executable Decision Graphs. Experiments show significant token savings while maintaining or improving success rates on benchmarks such as HumanEval and DeepSeek Harness.
Software engineering (SWE) agents resolve repository-level issues through long trajectories that grow increasingly expensive as context accumulates. Failed runs tend to be longer and exhibit redundant exploration or looping, suggesting that some failures may be detectable before completion.
arXiv:2608. 03222v1 Announce Type: cross Abstract: Software engineering (SWE) agents resolve repository-level issues through long trajectories that grow increasingly expensive as context accumulates.
Harness-Bench is a diagnostic benchmark that evaluates how different harness configurations affect LLM agent performance in realistic workflows. It tests 106 sandboxed offline tasks across multiple model backends, recording artifacts, execution traces, and usage statistics to analyze completion, process quality, efficiency, and failure behavior. The study finds significant variation in outcomes across model-harness pairings, indicating that agent capability should be reported at the configuration level rather than solely by the base model.
arXiv:2608. 15089v1 Announce Type: new Abstract: Long-horizon agents can fail even when their underlying models can solve the constituent steps.
LEASH is a training‑free framework that enables uncertainty‑aware escalation from a weak to a strong computer‑use agent. It uses latent uncertainty signals from the weak agent’s hidden states to decide when to hand off control, and a shadow observer to maintain oversight of the strong black‑box agent. Experiments on OSWorld show LEASH improves task success over standalone weak agents and outperforms random escalation, achieving comparable performance to Claude Opus 5 while cutting inference cost by 19.2%.
arXiv:2607. 20488v1 Announce Type: new Abstract: Multi-agent LLM frameworks typically fix their team topology at boot time.
Mingbird is a local‑first agent harness designed for small open‑weight language models (2–9 B) that run on ordinary laptops. It introduces ten mechanisms—such as a byte‑level net‑zero prefill budget, a finish gate that re‑reads the task before accepting completion, and signature‑level loop detection—to address common failure modes that arise from the harness rather than the model itself. In controlled experiments on the LRAB benchmark and the $ au^2$‑bench, Mingbird achieves higher overall scores (0.886 and 0.856 respectively) compared to other harnesses, and its ablation studies show that each mechanism contributes measurable performance gains.
Mid‑Harness proposes a test‑time compute strategy that samples and verifies candidate actions before execution, keeping the underlying generator and harness unchanged. Experiments show that with a strong verifier, sampling more actions significantly boosts success rates—e.g., a GPT‑5.6 verifier raises Pass@1 from 50.00 % to 68.03 % on TerminalBench‑Lite using eight samples. The approach also improves performance across various models, benchmarks, and harnesses, demonstrating that action scaling is a promising target for enhancing terminal agent reliability.