The paper introduces DCFA, a training‑free framework for attributing failures in large language model‑based multi‑agent systems. DCFA uses a global module to build causal‑inspired dependency graphs from system traces, pinpointing the earliest decisive error, and a local module that refines this attribution through counterfactual reasoning. Experiments on the Who&When benchmark across six LLMs demonstrate that DCFA improves step‑level accuracy by up to 8.27% over existing baselines.
By Zehao Wang, Lanjun Wang, Shilong Jin, Junjie Chen, Yanghua Xiao
The paper introduces Adaptive Influence Graphs (AIGs), a two‑stage framework that first converts a failed trace into a structured graph and then navigates it to pinpoint the critical error in multi‑agent large language model systems. Experiments across multiple models demonstrate that richer trace representations and adaptive graph construction improve failure attribution, with AIGs achieving state‑of‑the‑art results on the Who&When benchmark. The study shows that both the diagnosing model and the way traces are represented and explored are crucial for accurate failure attribution.
By Yarden Bakish, Amir Dudai, Roy Ganz, Oren Nuriel, Elad Ben Avraham, Mor Shpigel Nacson, Ron Litman
arXiv:2608.29646v1 Announce Type: new
Abstract: Large language model (LLM)-based agents have shown strong potential in solving complex tasks through multi-step reasoning, yet they remain vulnerable t...
By Jiayi Zhang, Zexin Wang, Degang Sun, Changhua Pei, Fei Sun, Gaogang Xie, Jingjing Li
arXiv:2607. 09996v1 Announce Type: new Abstract: Automated failure attribution uses LLMs to identify where and why agentic systems fail.
By Jiale Liu, Huajun Xi, Shaokun Zhang, Yifan Zeng, Tianwei Yue, Chi Wang, Jian Kang, Qingyun Wu, Huazheng Wang
arXiv:2606. 00765v1 Announce Type: new Abstract: LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication.
By Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim, Zhijie Wang, Tse-Hsun Chen
arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.
By Yufei Xia, Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang
arXiv:2606. 09071v1 Announce Type: new Abstract: Large language model (LLM) agents now solve complex tasks through long plan-and-execution traces, yet the ability to locate errors in a completed traces still lags far behind, especially in the \emph{silent failure} regime.
By Xiaofeng Lin, Yingxu Wang, Tung Sum Thomas Kwok, Daniel Guo, Sahil Arun Nale, Charles Fleming, Guang Cheng
arXiv:2606. 02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation.
By I\~naki Dellibarda Varela, R. Sendra-Arranz, Pablo Romero-Sorozabal, J. M. Valverde-Garc\'ia, Annemarie F. Laudanski, \'Alvaro Guti\'errez, Eduardo Rocon, Manuel Cebrian
arXiv:2606. 03467v1 Announce Type: new Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks.
By Taiyu Zhu, Yifan Wu, Weilin Jin, Ying Li, Gang Huang
arXiv:2602.02475v2 Announce Type: replace
Abstract: AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy...
By Shraddha Barke, Arnav Goyal, Alind Khare, Avaljot Singh, Suman Nath, Chetan Bansal
arXiv:2608. 15242v1 Announce Type: new Abstract: When a long-horizon agent execution fails, outcome-level evaluation reveals the unsuccessful result but not where the decisive error entered the trajectory.
By Yunfei Zhang, Boyu Feng, Changhua Pei, Zexin Wang, Zhihuang Peng, Xinlong Liu, Hengyue Jiang, Difeng Ma, Jiayi Zhang, Yongzhou Yao, Yanan Zhao, Fei Sun, Yintong Huo, Zhaoyang Liu, Jingjing Li, Gaogang Xie, Dan Pei
The paper introduces AGENTSCOPE, a neuro‑symbolic method for diagnosing failures in large language model agents. It abstracts agent trajectories into structured representations and employs neural invariants to define behavior properties. Using LLM‑guided reasoning on these abstractions, AGENTSCOPE identifies both the failure step and its type, outperforming existing techniques on several datasets.
By Jiayi Bi, Yanjie Gao, Yuanmin Xie, Liqun Li, Tianyin Xu, Fan Yang, Mao Yang