UMAS is a parameter‑free, cost‑neutral framework for system‑level uncertainty quantification in multi‑agent large language model systems. It estimates the credibility of inter‑agent influence using each generation’s intrinsic confidence, modulates uncertainty updates at each node, and aggregates these into a system‑level estimate. Evaluations on Debate and DyLAN show UMAS outperforms state‑of‑the‑art baselines by an average of 10.54 AUROC points and improves hallucination detection and answer selection, boosting MAS accuracy by up to 12.6 points on specific tasks.
By Hanwen Li, Jinhao Duan, Xiaoshuang Shi, Yue Zhang, Tianlong Chen, Kaidi Xu, Chenxi Yuan
The paper introduces RUPA, a trajectory‑level uncertainty quantification framework for large language model agents. RUPA models an agent’s execution as a directed graph of reasoning states, tool interactions, and environment feedback, then propagates uncertainty across this graph to capture long‑range dependencies. Experiments on benchmarks such as τ‑2, Terminal‑Bench‑2, and GAIA show that RUPA outperforms existing methods, enabling earlier failure detection and more reliable agent execution.
By Zhengzhao Ma. Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
The paper introduces Interaction-Effect Propagation (IE-Prop), a model that estimates error risk in large language model (LLM) based multi‑agent systems by conditioning on whether upstream agents are correct or incorrect. IE‑Prop learns separate effects for each sender‑receiver interaction, combines them using the sender’s estimated error risk, and recursively propagates these risks through the execution graph. Experiments on three architectures and benchmarks show that IE‑Prop improves final‑node AUROC by about 0.05 points over baselines and that multi‑step risk tracking adds an additional 0.03 AUROC in sequential systems.
By Shreyan Ganguly, Souradeep Chattopadhyay, Qisai Liu, Soumik Sarkar
arXiv:2611.07140v1 Announce Type: cross
Abstract: Teams increasingly build software from cooperating large language model (LLM) agents, often assuming more agents mean more reliability. Yet the usual...
By Burkay Tun\c{c}t\"urk, Koorosh Aslansefat, Yiannis Papadopoulos, Bao Nguyen
The paper introduces EMFA, a method for pinpointing the decisive error in large‑language‑model‑based multi‑agent systems. EMFA builds a structured representation of a failed execution, models how errors cascade and loop, and then screens candidates with propagation awareness before verifying them counterfactually. On the Who&When benchmark, EMFA outperforms prior methods, improving step‑level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.
The paper introduces EMFA, a method for attributing failures in large language model-based multi-agent systems by identifying the decisive error—an agent–step pair whose correction would restore correct execution. EMFA builds a structured representation of the failed trajectory, models error propagation and persistent loops, and applies propagation-aware screening with counterfactual verification to pinpoint the decisive error. On the Who&When benchmark, EMFA outperforms prior methods, improving step-level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.
By Jiaqi Liao, Yuanzhao Zhai, Huanxi Liu, Xu Zhang, Zheming Zhuang, Dawei Feng, Bo Ding, Huaimin Wang