arXiv AI
A Layered Analysis of Disagreement And Answer Quality in Multi-Agent LLM Debate
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 10
The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge
arXiv:2606. 10296v1 Announce Type: cross Abstract: Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediate reasoning that debate is designed to produce.
arXiv AI
6d ago
MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate
MABPD (Multi‑Agent Bias Probing & Detection) is a training‑free pipeline that uses three specialized large language model agents to analyze news articles from complementary perspectives and resolve disagreements via a Structured Argument Debate (SAD) protocol. SAD imposes an asymmetric burden of proof—biased claims lacking grounded textual evidence receive zero weight—along with role‑weighted voting and post‑consensus verification, replacing task‑specific supervised decision boundaries. Ablation studies show that the debate module alone accounts for up to a 10.6‑point F1 gain, and on the BABE benchmark MABPD attains 83.4% macro F1, within 0.7 percentage points of the supervised state‑of‑the‑art, while achieving 75.0% zero‑shot accuracy on the SemEval 2019 HyperPartisan corpus.
arXiv AI
Jun 2
Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
arXiv:2601. 19921v2 Announce Type: replace-cross Abstract: Multi-agent debate (MAD) is widely used to improve large language model (LLM) performance through test-time scaling, yet recent work shows that vanilla MAD often underperforms simple majority vote despite higher computational cost.
arXiv AI
3d ago
How AI Models Manage Epistemic Authority: A Taxonomy and Comparative Analysis of Responses to User Disagreement
The paper introduces a taxonomy of six user challenge types and a four-layer framework to analyze how large language models respond to user disagreement. Using a dataset of 2,310 challenge scenarios and 32,340 responses from 14 models, the study finds that models often validate users (85%) while still maintaining their original claim (65%). It also reports that models frequently apologize (33%) and transfer authority in advice contexts, with significant variation across model types and task domains.
arXiv AI
Jun 2
Argument Collapse: LLMs Flatten Long-Form Public Debate
arXiv:2606. 01736v1 Announce Type: cross Abstract: As LLMs are increasingly used to draft public-facing arguments, they may flatten public debate by repeatedly introducing the same polished, plausible arguments.
arXiv Computation and Language
Sep 1
Beyond Consensus: Downward Bias and Role Asymmetry in Multi-Agent LLM Judges for Subjective Evaluation
arXiv:2608.30373v1 Announce Type: new Abstract: Multi-Agent Debate (MAD) has been widely adopted to improve LLM-based evaluation by prompting multiple agents to negotiate and reach a consensus. Howev...