The paper introduces DCFA, a training‑free framework for attributing failures in large language model‑based multi‑agent systems. DCFA uses a global module to build causal‑inspired dependency graphs from system traces, pinpointing the earliest decisive error, and a local module that refines this attribution through counterfactual reasoning. Experiments on the Who&When benchmark across six LLMs demonstrate that DCFA improves step‑level accuracy by up to 8.27% over existing baselines.
By Zehao Wang, Lanjun Wang, Shilong Jin, Junjie Chen, Yanghua Xiao
The paper introduces Adaptive Influence Graphs (AIGs), a two‑stage framework that first converts a failed trace into a structured graph and then navigates it to pinpoint the critical error in multi‑agent large language model systems. Experiments across multiple models demonstrate that richer trace representations and adaptive graph construction improve failure attribution, with AIGs achieving state‑of‑the‑art results on the Who&When benchmark. The study shows that both the diagnosing model and the way traces are represented and explored are crucial for accurate failure attribution.
By Yarden Bakish, Amir Dudai, Roy Ganz, Oren Nuriel, Elad Ben Avraham, Mor Shpigel Nacson, Ron Litman
arXiv:2610.18780v1 Announce Type: new
Abstract: Large language model multi-agent systems extend individual agents' capabilities through collaboration across specialized roles, tools, and workflows. Y...
By Xiaoguang Guo, Yaohao Long, Chuxu Zhang
arXiv:2608.29646v1 Announce Type: new
Abstract: Large language model (LLM)-based agents have shown strong potential in solving complex tasks through multi-step reasoning, yet they remain vulnerable t...
By Jiayi Zhang, Zexin Wang, Degang Sun, Changhua Pei, Fei Sun, Gaogang Xie, Jingjing Li
DeFA is a dependency-guided framework that attributes failures in large language model agents by constructing an event dependency graph and a failure propagation graph from protocol relations and semantic dependencies. It identifies violating events, traces their sources and effects, and determines the decisive error, responsible agent, and error category. The method supports long trajectories through segmentation and has shown superior accuracy on text, image, and video tasks, while its diagnostic feedback can improve agent performance on subsequent tasks.
By Bo Deng, Xinlei Zheng, Yi Wei, Kang Zhou, Chongyang Tao, Renzhao Liang, Xuanren Chen, Lifan Guo, Chi Zhang
arXiv:2607. 09996v1 Announce Type: new Abstract: Automated failure attribution uses LLMs to identify where and why agentic systems fail.
By Jiale Liu, Huajun Xi, Shaokun Zhang, Yifan Zeng, Tianwei Yue, Chi Wang, Jian Kang, Qingyun Wu, Huazheng Wang
The paper introduces SOAP, a method for attributing failures in large language model agents without requiring step-level labels or a separate set of successful trajectories. SOAP builds a spectral reference space from LLM representations to assign base error scores to each step, then uses attention-based soft dependencies to propagate downstream error signals back to their likely upstream causes. Experiments demonstrate that SOAP outperforms existing baselines in accurately identifying decisive steps responsible for task failures.
By Do Nguyen-Thanh, Junlin Fang, Bing Li, Zhen Fang, Sean Du
arXiv:2606. 00765v1 Announce Type: new Abstract: LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication.
By Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim, Zhijie Wang, Tse-Hsun Chen
arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.
By Yufei Xia, Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang
The paper introduces EMFA, a method for attributing failures in large language model-based multi-agent systems by identifying the decisive error—an agent–step pair whose correction would restore correct execution. EMFA builds a structured representation of the failed trajectory, models error propagation and persistent loops, and applies propagation-aware screening with counterfactual verification to pinpoint the decisive error. On the Who&When benchmark, EMFA outperforms prior methods, improving step-level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.
By Jiaqi Liao, Yuanzhao Zhai, Huanxi Liu, Xu Zhang, Zheming Zhuang, Dawei Feng, Bo Ding, Huaimin Wang
The paper introduces EMFA, a method for pinpointing the decisive error in large‑language‑model‑based multi‑agent systems. EMFA builds a structured representation of a failed execution, models how errors cascade and loop, and then screens candidates with propagation awareness before verifying them counterfactually. On the Who&When benchmark, EMFA outperforms prior methods, improving step‑level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.
arXiv:2610.28811v1 Announce Type: new
Abstract: Agentic systems have demonstrated strong capabilities on complex tasks by enabling multiple interacting large language model (LLM)-based agents to coll...
By Wendy Zheng, Liang Wu, Yinhan He, Mayank Darbari, Jundong Li, Liangjie Hong