← Back to all news
arXiv AI By Shreyan Ganguly, Souradeep Chattopadhyay, Qisai Liu, Soumik Sarkar

Estimation of Propagated Risk in LLM-based Multi-Agent Systems

Read the original on arXiv AI →

The paper introduces Interaction-Effect Propagation (IE-Prop), a model that estimates error risk in large language model (LLM) based multi‑agent systems by conditioning on whether upstream agents are correct or incorrect. IE‑Prop learns separate effects for each sender‑receiver interaction, combines them using the sender’s estimated error risk, and recursively propagates these risks through the execution graph. Experiments on three architectures and benchmarks show that IE‑Prop improves final‑node AUROC by about 0.05 points over baselines and that multi‑step risk tracking adds an additional 0.03 AUROC in sequential systems.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 9

Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems

The paper introduces EMFA, a method for attributing failures in large language model-based multi-agent systems by identifying the decisive error—an agent–step pair whose correction would restore correct execution. EMFA builds a structured representation of the failed trajectory, models error propagation and persistent loops, and applies propagation-aware screening with counterfactual verification to pinpoint the decisive error. On the Who&When benchmark, EMFA outperforms prior methods, improving step-level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.

By Jiaqi Liao, Yuanzhao Zhai, Huanxi Liu, Xu Zhang, Zheming Zhuang, Dawei Feng, Bo Ding, Huaimin Wang
llmsagentsbenchmarks
More like this →
Hugging Face Trending Papers
Oct 8

Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems

The paper introduces EMFA, a method for pinpointing the decisive error in large‑language‑model‑based multi‑agent systems. EMFA builds a structured representation of a failed execution, models how errors cascade and loop, and then screens candidates with propagation awareness before verifying them counterfactually. On the Who&When benchmark, EMFA outperforms prior methods, improving step‑level attribution accuracy by 3.45 and 4.40 percentage points on two subsets.

llmsagentsbenchmarks
More like this →
arXiv AI
Aug 18

From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents

The paper introduces RUPA, a trajectory‑level uncertainty quantification framework for large language model agents. RUPA models an agent’s execution as a directed graph of reasoning states, tool interactions, and environment feedback, then propagates uncertainty across this graph to capture long‑range dependencies. Experiments on benchmarks such as τ‑2, Terminal‑Bench‑2, and GAIA show that RUPA outperforms existing methods, enabling earlier failure detection and more reliable agent execution.

By Zhengzhao Ma. Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
llmsagentsbenchmarkssafety
More like this →
arXiv Computation and Language
Aug 25

PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systems

PropUQ-MAS is a framework for uncertainty quantification in large language model (LLM) multi‑agent systems that models the system as a communication‑structured graph. It estimates the reliability of each step by combining local uncertainty with uncertainty inherited from upstream messages, addressing the risk of error propagation in inter‑agent communication. Experiments show consistent improvements in UQ metrics, with average gains of +6.10% in AUROC and +47.58% in PRR.

By Yaokun Liu, Yifan Liu, Daniel Yue Zhang, Ruichen Yao, Zelin Li, Dong Wang
llmsagents
More like this →