PropUQ-MAS is a framework for uncertainty quantification in large language model (LLM) multi‑agent systems that models the system as a communication‑structured graph. It estimates the reliability of each step by combining local uncertainty with uncertainty inherited from upstream messages, addressing the risk of error propagation in inter‑agent communication. Experiments show consistent improvements in UQ metrics, with average gains of +6.10% in AUROC and +47.58% in PRR.
By Yaokun Liu, Yifan Liu, Daniel Yue Zhang, Ruichen Yao, Zelin Li, Dong Wang
arXiv:2611.07140v1 Announce Type: cross
Abstract: Teams increasingly build software from cooperating large language model (LLM) agents, often assuming more agents mean more reliability. Yet the usual...
By Burkay Tun\c{c}t\"urk, Koorosh Aslansefat, Yiannis Papadopoulos, Bao Nguyen
The paper introduces RUPA, a trajectory‑level uncertainty quantification framework for large language model agents. RUPA models an agent’s execution as a directed graph of reasoning states, tool interactions, and environment feedback, then propagates uncertainty across this graph to capture long‑range dependencies. Experiments on benchmarks such as τ‑2, Terminal‑Bench‑2, and GAIA show that RUPA outperforms existing methods, enabling earlier failure detection and more reliable agent execution.
By Zhengzhao Ma. Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
arXiv:2608. 14707v1 Announce Type: new Abstract: As large language model (LLM)-based multi-agent systems become increasingly capable, coordinating agents under uncertainty becomes a fundamental challenge.
By John Knowlton, Aritra Guha, Risto Miikkulainen
arXiv:2606. 19868v1 Announce Type: new Abstract: Although large language models (LLMs) have shown strong capabilities across a wide range of tasks, their outputs often remain unreliable and may contain hallucinations, making uncertainty estimation (UE) essential for building trustworthy LLMs.
By Jiayi Wang, Xu-Yao Zhang
The paper addresses the lack of system‑level confidence estimates in multiagent language model systems such as collaborative reasoning and debate. It introduces confidence composition methods, including confidence‑aware routing and log‑odds pooling, to combine agent confidences while maintaining selective utility and probabilistic reliability. Experiments on five benchmarks with diverse model pairs show that gated‑fusion techniques improve AUARC and Brier scores compared to single‑agent and standard debate baselines, and a shared dependence discount further enhances reliability.
By Ali Elahi, Michael J. Curry, Barbara Di Eugenio
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
arXiv:2607. 02186v1 Announce Type: new Abstract: Software development is a complex task that demands cooperation among agents with diverse roles.
By Temitayo Olamilekan Ogunsusi, Lijun Qian, Xishuang Dong
GraphACM introduces a graph-based framework to measure fine-grained action consistency in large language model agents. By merging equivalent states into a directed, action-labeled graph across multiple task rollouts, it enables comparisons of actions at shared decision points and defines node-level and global consistency metrics. Experiments on ALFWorld, AppWorld, and WebShop show that the global consistency score predicts task success with an AUROC of 0.811, outperforming prior consistency baselines and verbalized confidence.
By Yaxin Ma, Wenbo Zhang, Satya Krishna Pothapragada, Alina Zare
arXiv:2606. 09800v1 Announce Type: cross Abstract: Multi-agent code generation offers a promising paradigm for autonomous software development by simulating the human software engineering lifecycle.
By Shizhe Lin, Ladan Tahvildari
Behavioral consistency across repeated executions provides an informative signal for assessing the reliability of large language model (LLM) agents. However, existing approaches typically summarize co...
arXiv:2606. 03135v1 Announce Type: new Abstract: Large Language Model (LLM) agents often operate under underspecified user instructions, where latent uncertainty over user intent leads to erroneous tool actions.
By Mengyi Deng, Zhiwei Li, Xin Li, Tingyu Zhu, Ying Zhao, Zhijiang Guo, Wei Wang