EDGE is a framework that attributes multiple related errors in multi-agent large language model systems by constructing an error dependency graph from observed error events. It validates a reliable causal subset through counterfactual rollout and uses this inference graph to guide a two-stage LLM-as-judge detector for error attribution. Experiments on TRAIL and MAST demonstrate that EDGE improves category-level multi-error attribution across most models and settings, and that the graph aids explanation and repair analysis.
By Jun Hou, Priya Pitre, Yi Fang, Xuan Wang
arXiv:2610.28811v1 Announce Type: new
Abstract: Agentic systems have demonstrated strong capabilities on complex tasks by enabling multiple interacting large language model (LLM)-based agents to coll...
By Wendy Zheng, Liang Wu, Yinhan He, Mayank Darbari, Jundong Li, Liangjie Hong
arXiv:2606. 00765v1 Announce Type: new Abstract: LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication.
By Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim, Zhijie Wang, Tse-Hsun Chen
The paper introduces EMFA, a method for pinpointing the decisive error in large‑language‑model‑based multi‑agent systems. EMFA builds a structured representation of a failed execution, models how errors cascade and loop, and then screens candidates with propagation awareness before verifying them counterfactually. On the Who&When benchmark, EMFA outperforms prior methods, improving step‑level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.
The paper introduces EMFA, a method for attributing failures in large language model-based multi-agent systems by identifying the decisive error—an agent–step pair whose correction would restore correct execution. EMFA builds a structured representation of the failed trajectory, models error propagation and persistent loops, and applies propagation-aware screening with counterfactual verification to pinpoint the decisive error. On the Who&When benchmark, EMFA outperforms prior methods, improving step-level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.
By Jiaqi Liao, Yuanzhao Zhai, Huanxi Liu, Xu Zhang, Zheming Zhuang, Dawei Feng, Bo Ding, Huaimin Wang
The paper introduces SOAP, a method for attributing failures in large language model agents without requiring step-level labels or a separate set of successful trajectories. SOAP builds a spectral reference space from LLM representations to assign base error scores to each step, then uses attention-based soft dependencies to propagate downstream error signals back to their likely upstream causes. Experiments demonstrate that SOAP outperforms existing baselines in accurately identifying decisive steps responsible for task failures.
By Do Nguyen-Thanh, Junlin Fang, Bing Li, Zhen Fang, Sean Du