arXiv AI By Jiaming Zhang, Yuwan Liu, Yue Huang, Sisi Duan

MiniRep: Robust Reputation-Based Aggregation for Multi-Agent Debate

Read the original on arXiv AI →

MiniRep is a reputation‑based aggregation system designed for multi‑agent debate (MAD) that remains robust even when malicious agents are present. It evaluates agents on both their current task performance and historical reputation, while preventing groups of agents with highly similar responses from dominating the final decision. Experiments on the MATH benchmark show that MiniRep consistently outperforms conventional MAD aggregation and other reputation‑based approaches across a wide range of attack scenarios.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 1

MADBench: Benchmarking the Security of Multi-Agent Debate

MADBench is a benchmark that evaluates the security of Multi-Agent Debate (MAD) systems, which allow large language models to exchange and critique answers. The study categorizes attacks into a layered taxonomy aligned with the MAD workflow and tests six attack families across 356 source tasks and 3,958 test cases. Results indicate that while MAD can reduce answer accuracy attacks compared to single-agent baselines, it may amplify unauthorized reads or writes, and even with collusion among agents, the final answer changes from correct to wrong only 28.30% of the time.

By Yuwan Liu, Jiaming Zhang, Yue Huang, Sisi Duan
arXiv AI
Jun 12

Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents

arXiv:2606. 13385v1 Announce Type: cross Abstract: Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments, where they operate over untrusted web content and execute actions with direct consequences.

By Zihao Wang, Yiming Li, Yutong Wu, Zheyu Liu, Kangjie Chen, Fok Kar Wai, Pin-Yu Chen, Vrizlynn L. L. Thing, Bo Li, Dacheng Tao, Tianwei Zhang
arXiv AI
3d ago

POIROT: Agents as their own auditors for failure attribution in large language model multi-agent systems

POIROT (Phased Oversight via Introspection, Review and Opinion Tallying) is a modular protocol that lets agents in large language model multi‑agent systems attribute failures to themselves by casting private votes in a hazard space and aggregating those votes. The protocol progresses through phases—from independent self‑assessment to shared assessments and peer interrogation—to evaluate the contribution of each phase. Experiments on the Who&When benchmark and the newly released BLAME benchmark show that agent‑based attribution, especially the independent self‑assessment phase, outperforms a single‑LLM evaluator across multiple models and benchmarks.

By I\~naki Dellibarda Varela, R. Sendra-Arranz, Pablo Romero-Sorozabal, J. M. Valverde-Garc\'ia, Annemarie F. Laudanski, \'Alvaro Guti\'errez, Eduardo Rocon, Manuel Cebrian
arXiv AI
1d ago

Securing Multi-Agent Systems by Design

The paper introduces response manipulation attacks (ReMA) that iteratively optimize malicious agent responses to compromise multi‑agent system outputs. It proposes SecMAS, a secure aggregation rule that selects responses most consistent with a large subset of others, providing theoretical security guarantees against both untargeted and targeted attacks. Experiments show existing aggregation rules are vulnerable to ReMA, while SecMAS defends effectively without sacrificing utility.

By Xilong Wang, Xiaoxing Chen, Neil Gong