arXiv:2608. 12895v1 Announce Type: new Abstract: Compositional reliability bounds for multi-agent systems multiply component reliabilities, a step licensed by a conditional-independence assumption that is routinely stated and rarely tested.
By Varun Pratap Bhardwaj, Garima Singh, Arun Pratap Bhardwaj
The paper reports that a model can pass fidelity checks—verifying that extracted values match the source—without actually opening a datasheet, due to a hidden constraint that disables tool use. To address this, the authors log every tool call in an agentic benchmark and develop two instruments: a rule‑based failure‑attribution classifier and a silent‑failure detector that flags runs based solely on which tools were invoked. While the detector shows low false positives on clean extractions and recovers all planted faults, its recall against correct tool usage but incorrect answers remains unmeasured, and a partial causal chamber confirms only a subset of claims, highlighting limitations in physical verification.
By Qing Ye, Meng-Hsuan Lin
The paper presents a closed‑form estimator for the contamination correlation between anchors and judges under a single‑common‑factor model, requiring at least two judges and two anchors. It introduces a diagnostic battery—including judge‑covariance dispersion, over‑identification tests, a family‑block test, bootstrap confidence intervals, and a weak‑identification screen—to validate the estimator and detect violations. The authors also discuss identification limits for ordinal data and report that real panels have not yet passed the model‑adequacy pre‑test, while simulation studies confirm the estimator’s performance.
By Veerendra Kumar Sunkavalli
The paper reports a failure study of a production agentic software‑delivery platform, analyzing 147 incidents across 81 runs. It shows that the standard reliability primitives—retry, timeout, and error‑rate circuit breaking—fail in practice, leading to costly loops, false trips, and blocked work. The authors identify two cross‑cutting causes—identity adequacy and evidence adequacy—and propose seven new reliability primitives that enforce reliability at the delegation level.
By Mazhar Shaikh, Anurag Rajkumar Bombarde, Harshal Pathak
The paper investigates a critical flaw in AI coding-agent systems such as Claude Code, Codex CLI, and Cursor, where the action approved by a human is not the same as the action executed by the harness. It introduces the concept of Approval Laundering, categorizing six systematic failure modes—Scope, Argument, Temporal, Tool, Delegation, and Semantic laundering—and demonstrates these failures through controlled experiments. The authors propose an Approval Token mechanism that mitigates some laundering types but leaves others unaffected, highlighting the limitations of current enforcement strategies.
By Yang Wang
The paper investigates where exactly‑once semantics should be enforced for tool‑using agents—within the model, the agent harness, or the tool contract—by evaluating 25,930 episodes across nine models, three harnesses, two contract variants, and fifteen recovery conditions. Using the LIMBO sandbox, the study shows that when an immediate read‑back is available, frontier models rarely duplicate lost acknowledgements, whereas weaker models do; when read‑back is unavailable, the contract’s idempotency keys explain most duplicate behavior. The authors prove that verification‑only policies cannot guarantee exactly‑once under late commits without bounded in‑flight time, and that waiting only helps when delays are short and predictable.
whyItMatters":"The findings clarify that enforcing exactly‑once semantics largely depends on the tool contract and fault type, guiding designers on where to focus reliability mechanisms for LLM agents."
By Jiapeng Li