arXiv Machine Learning
A Mechanistic Study of Automated Failure Attribution in LLM-based Agentic Systems
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 3
StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems
arXiv:2606. 03467v1 Announce Type: new Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks.
arXiv AI
Sep 7
DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems
The paper introduces DCFA, a training‑free framework for attributing failures in large language model‑based multi‑agent systems. DCFA uses a global module to build causal‑inspired dependency graphs from system traces, pinpointing the earliest decisive error, and a local module that refines this attribution through counterfactual reasoning. Experiments on the Who&When benchmark across six LLMs demonstrate that DCFA improves step‑level accuracy by up to 8.27% over existing baselines.
arXiv AI
Jul 14
AgentAbstain: Do LLM Agents Know When Not to Act?
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.
arXiv AI
Oct 9
Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
The paper introduces EMFA, a method for attributing failures in large language model-based multi-agent systems by identifying the decisive error—an agent–step pair whose correction would restore correct execution. EMFA builds a structured representation of the failed trajectory, models error propagation and persistent loops, and applies propagation-aware screening with counterfactual verification to pinpoint the decisive error. On the Who&When benchmark, EMFA outperforms prior methods, improving step-level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.
arXiv Machine Learning
Oct 15
MCAIR: Monitoring and Correcting Agent Unfaithfulness via Internal Representations
arXiv:2610.19679v1 Announce Type: cross Abstract: Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in completing complex tasks through multi-step reasoning and tool i...
arXiv AI
Sep 15
Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
The paper introduces Continual Search, an iterative framework that guides large language models to persistently search for diagnostic evidence in long AI agent execution logs, addressing the limitations of one-shot judgments. Evaluated on four existing RCA benchmarks and a new large-scale dataset called MegaRCA-Mix, Continual Search consistently boosts attribution performance, achieving a 40% F1 improvement for GPT‑5.5 on MegaRCA‑Mix. The results show that effective search can outweigh raw model scale, enabling lower-tier models to outperform higher-tier ones in root‑cause attribution tasks.