arXiv AI
More Agents, More Reliable? System-Level Reliability of Multi-Agent LLMs under Operational-Profile Uncertainty
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 27
UMAS: System-Level Uncertainty Quantification for Multi-Agent LLM Systems
UMAS is a parameter‑free, cost‑neutral framework for system‑level uncertainty quantification in multi‑agent large language model systems. It estimates the credibility of inter‑agent influence using each generation’s intrinsic confidence, modulates uncertainty updates at each node, and aggregates these into a system‑level estimate. Evaluations on Debate and DyLAN show UMAS outperforms state‑of‑the‑art baselines by an average of 10.54 AUROC points and improves hallucination detection and answer selection, boosting MAS accuracy by up to 12.6 points on specific tasks.
arXiv AI
Jul 29
Runtime Uncertainty Monitoring for LLM-Based Multi-Agent Systems Using Bayesian Networks
arXiv:2607. 25877v1 Announce Type: new Abstract: This paper investigates how multi-agent systems (MAS)-based on large language models (LLMs) can support actuarial risk modelling, with a particular focus on uncertainty quantification.
arXiv AI
Jul 10
Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems
arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.
arXiv AI
Jun 2
POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems
arXiv:2606. 02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation.
arXiv AI
Jun 3
Towards a Science of AI Agent Reliability
arXiv:2602. 16666v3 Announce Type: replace Abstract: AI agents are increasingly deployed to execute important tasks.
arXiv Computation and Language
Oct 27
VerifyMAS: Hypothesis Verification for Failure Attribution in LLM-based Multi-Agent Systems
arXiv:2605.17467v2 Announce Type: replace Abstract: Large language model-driven multi-agent systems (LLM-MAS) excel at complex tasks, yet unreliable agents remain a key bottleneck to system-level rel...