arXiv AI
A Layered Analysis of Disagreement And Answer Quality in Multi-Agent LLM Debate
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
1d ago
Who Convinced Whom? Tracing How LLMs Change Their Minds in Multi-Agent Systems
arXiv:2610.35930v1 Announce Type: cross Abstract: Multi-agent debate (MAD) aims to improve language-model decisions through the exchange of arguments, yet agreement after debate need not reflect inde...
arXiv Computation and Language
Oct 1
Multi-agent discussion gains less when dissent is withheld
arXiv:2609.38324v1 Announce Type: cross Abstract: Multi-agent systems of LLMs add discussion to majority voting and are therefore expected to be more capable. However, empirical reports conflict on w...
arXiv AI
Sep 30
Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models
The study evaluates multi‑agent debate (MAD) in small language models, testing whether cognitive diversity—via personas, sampling temperature, or model identity—drives performance gains. Across 23 models, five tasks, and over 5,500 runs, MAD consistently outperforms single‑model inference but, when matched for generation budget, it ties or falls behind self‑consistency sampling, with persona prompting actually reducing accuracy. The authors find that MAD’s benefits largely stem from the first answer exchange and that many reported gains are due to ensemble‑sampling effects rather than true diversity, highlighting the need for budget‑matched, contamination‑checked baselines. whyItMatters":"The findings clarify that MAD’s perceived advantages may be overestimated and that future debate mechanisms must be evaluated against rigorous, budget‑matched baselines to ensure genuine performance improvements."
arXiv Computation and Language
Oct 5
Silent Dissent: LLM Agents That Yield to the Majority Still Represent Their Original Premise
The paper investigates whether large language model (LLM) agents that yield to a unanimous majority in multi‑agent debate truly change their underlying premise or merely adjust their statement. Using two‑hop factual questions with an unstated intermediate entity, the authors show that agents that concede still encode the original bridge in their internal representations, as revealed by a Jacobian‑lens analysis, even when the majority answer is wrong. Experiments across several open‑weight models demonstrate that hiding or removing the agent’s earlier answer increases conformity and that the original premise can be recovered from the question alone.
arXiv AI
Jun 10
The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge
arXiv:2606. 10296v1 Announce Type: cross Abstract: Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediate reasoning that debate is designed to produce.
arXiv AI
Sep 7
MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate
MABPD (Multi‑Agent Bias Probing & Detection) is a training‑free pipeline that uses three specialized large language model agents to analyze news articles from complementary perspectives and resolve disagreements via a Structured Argument Debate (SAD) protocol. SAD imposes an asymmetric burden of proof—biased claims lacking grounded textual evidence receive zero weight—along with role‑weighted voting and post‑consensus verification, replacing task‑specific supervised decision boundaries. Ablation studies show that the debate module alone accounts for up to a 10.6‑point F1 gain, and on the BABE benchmark MABPD attains 83.4% macro F1, within 0.7 percentage points of the supervised state‑of‑the‑art, while achieving 75.0% zero‑shot accuracy on the SemEval 2019 HyperPartisan corpus.