UMAS is a parameter‑free, cost‑neutral framework for system‑level uncertainty quantification in multi‑agent large language model systems. It estimates the credibility of inter‑agent influence using each generation’s intrinsic confidence, modulates uncertainty updates at each node, and aggregates these into a system‑level estimate. Evaluations on Debate and DyLAN show UMAS outperforms state‑of‑the‑art baselines by an average of 10.54 AUROC points and improves hallucination detection and answer selection, boosting MAS accuracy by up to 12.6 points on specific tasks.
By Hanwen Li, Jinhao Duan, Xiaoshuang Shi, Yue Zhang, Tianlong Chen, Kaidi Xu, Chenxi Yuan
arXiv:2607. 25877v1 Announce Type: new Abstract: This paper investigates how multi-agent systems (MAS)-based on large language models (LLMs) can support actuarial risk modelling, with a particular focus on uncertainty quantification.
By Bart Custers, Koorosh Aslansefat
arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.
By Yufei Xia, Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang
arXiv:2606. 02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation.
By I\~naki Dellibarda Varela, R. Sendra-Arranz, Pablo Romero-Sorozabal, J. M. Valverde-Garc\'ia, Annemarie F. Laudanski, \'Alvaro Guti\'errez, Eduardo Rocon, Manuel Cebrian
arXiv:2602. 16666v3 Announce Type: replace Abstract: AI agents are increasingly deployed to execute important tasks.
By Stephan Rabanser, Sayash Kapoor, Peter Kirgis, Kangheng Liu, Saiteja Utpala, Arvind Narayanan
arXiv:2605.17467v2 Announce Type: replace
Abstract: Large language model-driven multi-agent systems (LLM-MAS) excel at complex tasks, yet unreliable agents remain a key bottleneck to system-level rel...
By Hezhe Qiao, Hanghang Tong, Ee-Peng Lim, Bing Liu, Guansong Pang
The paper introduces Bayesian Dialectical Argumentation (BDA), a method for aggregating answers from multiple large language models (LLMs) in a council setting. BDA treats each LLM’s typed moves—proposals, challenges, and concessions—as evidence in a classical annotator model, estimating per-agent reliability even when some agents are persistently unreliable. By weighting evidence according to these inferred reliabilities, BDA produces calibrated posterior probabilities for candidate answers and can invert unreliable agents instead of merely outvoting them, achieving superior calibration and robustness on both binary and multi-class benchmarks without extra LLM calls.
By Ionel Eduard Stan, Paolo Napoletano
The paper introduces Interaction-Effect Propagation (IE-Prop), a model that estimates error risk in large language model (LLM) based multi‑agent systems by conditioning on whether upstream agents are correct or incorrect. IE‑Prop learns separate effects for each sender‑receiver interaction, combines them using the sender’s estimated error risk, and recursively propagates these risks through the execution graph. Experiments on three architectures and benchmarks show that IE‑Prop improves final‑node AUROC by about 0.05 points over baselines and that multi‑step risk tracking adds an additional 0.03 AUROC in sequential systems.
By Shreyan Ganguly, Souradeep Chattopadhyay, Qisai Liu, Soumik Sarkar
PropUQ-MAS is a framework for uncertainty quantification in large language model (LLM) multi‑agent systems that models the system as a communication‑structured graph. It estimates the reliability of each step by combining local uncertainty with uncertainty inherited from upstream messages, addressing the risk of error propagation in inter‑agent communication. Experiments show consistent improvements in UQ metrics, with average gains of +6.10% in AUROC and +47.58% in PRR.
By Yaokun Liu, Yifan Liu, Daniel Yue Zhang, Ruichen Yao, Zelin Li, Dong Wang
The paper presents a taxonomy of communication failures in multi-agent large language model systems, derived from 3,211 failed multi-round trajectories across diverse domains such as medical QA, coding, finance, multi-hop reasoning, ethics, and embodied text tasks. It analyzes how failures arise, noting that they are largely driven by recovery/correction and consensus/decision issues rather than isolated first-round errors, and highlights patterns like early convergence, minority-signal suppression, and wrong framing persistence. The authors recommend stronger disagreement handling, evidence verification, minority-signal preservation, and recovery checks before finalizing decisions.
By Priya Pitre, Jun Hou, Yi Fang, Naren Ramakrishnan, Xuan Wang
arXiv:2604. 02863v2 Announce Type: replace Abstract: Majority voting is the standard for aggregating multi-agent responses into a final decision.
By Yiqing Liu, Hantao Yao, Wu Liu, Yongdong Zhang
Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons. Reported benchmark gains often obscure recurring failure modes documented across otherwise unrelated evaluation efforts.