PropUQ-MAS is a framework for uncertainty quantification in large language model (LLM) multi‑agent systems that models the system as a communication‑structured graph. It estimates the reliability of each step by combining local uncertainty with uncertainty inherited from upstream messages, addressing the risk of error propagation in inter‑agent communication. Experiments show consistent improvements in UQ metrics, with average gains of +6.10% in AUROC and +47.58% in PRR.
By Yaokun Liu, Yifan Liu, Daniel Yue Zhang, Ruichen Yao, Zelin Li, Dong Wang
arXiv:2611.07140v1 Announce Type: cross
Abstract: Teams increasingly build software from cooperating large language model (LLM) agents, often assuming more agents mean more reliability. Yet the usual...
By Burkay Tun\c{c}t\"urk, Koorosh Aslansefat, Yiannis Papadopoulos, Bao Nguyen
The paper introduces RUPA, a trajectory‑level uncertainty quantification framework for large language model agents. RUPA models an agent’s execution as a directed graph of reasoning states, tool interactions, and environment feedback, then propagates uncertainty across this graph to capture long‑range dependencies. Experiments on benchmarks such as τ‑2, Terminal‑Bench‑2, and GAIA show that RUPA outperforms existing methods, enabling earlier failure detection and more reliable agent execution.
By Zhengzhao Ma. Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
arXiv:2608. 14707v1 Announce Type: new Abstract: As large language model (LLM)-based multi-agent systems become increasingly capable, coordinating agents under uncertainty becomes a fundamental challenge.
By John Knowlton, Aritra Guha, Risto Miikkulainen
arXiv:2606. 19868v1 Announce Type: new Abstract: Although large language models (LLMs) have shown strong capabilities across a wide range of tasks, their outputs often remain unreliable and may contain hallucinations, making uncertainty estimation (UE) essential for building trustworthy LLMs.
By Jiayi Wang, Xu-Yao Zhang
The paper addresses the lack of system‑level confidence estimates in multiagent language model systems such as collaborative reasoning and debate. It introduces confidence composition methods, including confidence‑aware routing and log‑odds pooling, to combine agent confidences while maintaining selective utility and probabilistic reliability. Experiments on five benchmarks with diverse model pairs show that gated‑fusion techniques improve AUARC and Brier scores compared to single‑agent and standard debate baselines, and a shared dependence discount further enhances reliability.
By Ali Elahi, Michael J. Curry, Barbara Di Eugenio