arXiv:2608.27808v1 Announce Type: cross
Abstract: Self-report is the cheapest oversight channel a deployer has, and on capable computer-use agents (CUAs) it fails precisely where oversight matters. O...
By Divake Kumar, Sina Tayebati, Devashri Naik, Amanda Sofie Rios, Nilesh Ahuja, Omesh Tickoo, Ranganath Krishnan, Amit Ranjan Trivedi
The paper introduces a coroutine-bridge harness that lets a language model emit a Python program to manage tool calls in the CAR-bench evaluation. By decoupling model invocations from tool round-trips, the approach reduces model calls to a median of two per task while maintaining seven agent turns, achieving a median latency of 1.8 s on a Cerebras gpt‑oss‑120b. The harness achieved 60.0 % Pass³ on the official hidden evaluation, outperforming the baseline by 4.5× and matching frontier-model agents on GPT‑5.5, all while keeping the prompt largely cached and minimizing input compute.
By Ivan Matveev
The paper introduces prefix executability, a trajectory-level evaluation protocol for tool-using agents that checks whether every prefix of a predicted tool-call sequence remains executable under tool schemas, state preconditions, and data dependencies. It defines metrics such as Prefix Executability Curves, First-Error Depth, Trajectory Executability Rate, horizon-normalized AUC-PEC, and Recovery Rate to quantify reliability. Experiments on BFCL-v3, ToolSandbox, and $ au^3$-bench show that agents can achieve high final success while having low trajectory executability, revealing failure modes that final success metrics miss, and demonstrate that explicit pre-execution plan checking can improve prefix-level reliability without sacrificing terminal success.
By Amir H. Rezaeian, Weiyi Sun, Yassine Benajiba, Dan Roth
HarnessMM (Harness Minus Minus) is a method that improves harness token efficiency by using guarded decision reuse and recursive self‑improvement (RSI). It refines harness configurations from training feedback, reuses resolved decisions to replace later reasoning callbacks, and stores actions in Executable Decision Graphs. Experiments show significant token savings while maintaining or improving success rates on benchmarks such as HumanEval and DeepSeek Harness.
By Yuhua Jiang, Chang Xu, Feifei Gao, Li Zhao
Software engineering (SWE) agents resolve repository-level issues through long trajectories that grow increasingly expensive as context accumulates. Failed runs tend to be longer and exhibit redundant exploration or looping, suggesting that some failures may be detectable before completion.
arXiv:2608. 03222v1 Announce Type: cross Abstract: Software engineering (SWE) agents resolve repository-level issues through long trajectories that grow increasingly expensive as context accumulates.
By Chenyu Wang, Yunbo Lyu, Junda He, Zhou Yang, Chenxing Zhong, Yaniv Harel, David Lo