arXiv AI
Structural Baselines Matter for Agent Failure Detection and Attribution
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 10
From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents
arXiv:2606. 09863v1 Announce Type: new Abstract: LLM agents can fail silently by asserting task completion when the environment state shows otherwise.
arXiv AI
4d ago
Auditing Failure Attribution: Label Predictability and Judge Behavior
arXiv:2610.17747v1 Announce Type: cross Abstract: Failure-attribution benchmarks ask an LLM judge which step or agent caused a failure. A high score should mean the judge checked the faulty step agai...
Hugging Face Trending Papers
6d ago
Executable Prefix Values for Locating Agent Failures
The paper investigates whether human or LLM‑judge‑identified ‘first mistake’ points truly indicate failure causes in LLM agents. By reconstructing intermediate states and computing a prefix value V(h_t) for each step, the authors identify a certified point of no return (PNR) and compare it to judge‑labeled steps. Their experiments on 150 SWE‑bench tasks show that judge labels often miss the decisive step, while restarting at the measured PNR improves success rates from 0.44 to 0.57.
arXiv AI
4d ago
Executable Prefix Values for Locating Agent Failures
arXiv:2610.17633v1 Announce Type: new Abstract: Work on debugging LLM agents treats the "first mistake" marked by a human or LLM judge as ground truth for where a trajectory failed. We ask whether su...
arXiv Machine Learning
Sep 1
CatchBench: When Can an Agent Failure Be Caught?
arXiv:2608.22808v2 Announce Type: replace Abstract: When can an agent failure be caught? An audit is usually limited by the record rather than by the method. CatchBench therefore puts one auditor's q...
arXiv AI
Jun 2
Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability
arXiv:2606. 01365v1 Announce Type: new Abstract: Tool-using multi-agent large language model (LLM) systems spend computation through model tokens, tool calls, retries, and code execution before producing an answer.