arXiv AI

MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate

MABPD (Multi‑Agent Bias Probing & Detection) is a training‑free pipeline that uses three specialized large language model agents to analyze news articles from complementary perspectives and resolve disagreements via a Structured Argument Debate (SAD) protocol. SAD imposes an asymmetric burden of proof—biased claims lacking grounded textual evidence receive zero weight—along with role‑weighted voting and post‑consensus verification, replacing task‑specific supervised decision boundaries. Ablation studies show that the debate module alone accounts for up to a 10.6‑point F1 gain, and on the BABE benchmark MABPD attains 83.4% macro F1, within 0.7 percentage points of the supervised state‑of‑the‑art, while achieving 75.0% zero‑shot accuracy on the SemEval 2019 HyperPartisan corpus.

arXiv Computation and Language

Sep 10

Who Argues What? Joint Argument-Entity Detection and Classification in Political Debates

The paper introduces DNE‑ElecDeb, an enriched version of the USElecDeb dataset that annotates Debate Named Entities (DNEs) in both argumentative and non‑argumentative spans, and defines Debate Named Entity Recognition (DNER) as a new task. It proposes Joint Argument and Entity Tagging (JAET), a generative framework that fine‑tunes decoder‑only LLMs to insert inline argument and entity tags into debate turns while preserving the original transcript. JAET achieves significant improvements in joint AM+DNER performance (+27.3% relative F1 in the untyped setting and +41.9% in the typed setting) over sequential pipelines, and these gains generalize to Persuasive Essays (+26.6% and +52.7%).

By Lucio La Cava, Stefano Francesco Monea, Sergio Greco

llms nlp

Hugging Face Trending Papers

Sep 9

Who Argues What? Joint Argument-Entity Detection and Classification in Political Debates

Political debates are often analyzed through Argument Mining (AM) to investigate the key arguments that drive them. However, political arguments are rarely interpretable from argumentative spans alone...

llms nlp

arXiv AI

Sep 1

JudgePanel: A Compact Judge with Panel Deliberation via Adaptive Multi-Reward Reinforcement Learning

arXiv:2608.29168v1 Announce Type: new Abstract: The LLM-as-a-Judge paradigm has emerged as a scalable alternative to human evaluation. However, single-model judges are limited by their inherent model...

By Yiyue Qian, Shinan Zhang, Huan Song, Hannah Marlowe

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Sep 10

A Layered Analysis of Disagreement And Answer Quality in Multi-Agent LLM Debate

arXiv:2609.08016v1 Announce Type: new Abstract: Multi-agent debate, in which several LLMs exchange arguments before answering, is widely assumed to improve answer quality by surfacing genuine disagre...

By Chen Qian

llms agents safety

arXiv Machine Learning

Jun 25

Diagnosing and Mitigating Compounding Failures in Agentic Persuasion via Taxonomic Strategy Retrieval

arXiv:2606. 24976v1 Announce Type: cross Abstract: Foundation-model agents in multi-step, open-ended environments frequently suffer from compounding errors, where early mistakes contaminate long-horizon trajectories.

By Pradyumna Narayana, Sana Ayromlou, Purvi Sehgal

rag agents

arXiv AI

Oct 21

BAIT: Can LLMs Find What Biases a Reader? A Benchmark for Cognitive Bias Triggers in News

arXiv:2610.28092v1 Announce Type: cross Abstract: Persuasive writing exploits cognitive biases, recruiting a reader's mental shortcuts to shape belief, often without the reader noticing. We call the...

By Yousra Fettach, Tijl De Bie

llms benchmarks safety

arXiv AI

Jun 12

Multiagent Protocols with Aggregated Confidence Signals

arXiv:2606. 13591v1 Announce Type: new Abstract: Confidence is used for reliability, oversight, and a range of downstream decision tasks in Natural Language Processing (NLP), yet no existing method produces or evaluates a confidence for the output of a multiagent system.

By Ali Elahi, Barbara Di Eugenio

agents nlp benchmarks

arXiv AI

Aug 14

Unmasking Conversational Bias in AI Multiagent Systems

arXiv:2501. 14844v3 Announce Type: replace-cross Abstract: Detecting biases in the outputs produced by generative models is essential to reduce the potential risks associated with their application in critical settings.

By Erica Coppolillo, Giuseppe Manco, Luca Maria Aiello

llms agents benchmarks safety

arXiv Computation and Language

Oct 9

The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection

The paper introduces MARS-Gov, a standpoint-aware multi-agent system designed to detect and mitigate bias in Dutch government documents. It combines legal evidence retrieval, open-set target screening, specialized jurors, conservative routing, and rewrite verification, and adds a dynamic "10th juror" when new bias groups are identified. On the DGDB benchmark, MARS-Gov achieves state‑of‑the‑art performance with an F1 score of 0.880, surpassing zero‑shot LLM detectors and supervised Dutch encoders while reducing unnecessary interventions to 2.5%.

By Yuchen Miao, Zijun Wang, Chang Han, Yurui Shi, Mingtai Zhang, Siyang Xu

llms agents benchmarks safety

arXiv AI

Jun 17

RooseBERT: A New Deal For Political Language Modelling

arXiv:2508. 03250v4 Announce Type: replace-cross Abstract: The increasing amount of political debates and politics-related discussions calls for the definition of novel computational methods to automatically analyse such content with the final goal of lightening up political deliberation to citizens.

By Deborah Dore, Elena Cabrio, Serena Villata

llms nlp fine-tuning

arXiv Computation and Language

Oct 1

JuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation

arXiv:2609.40103v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as automated judges for AI-generated content, yet a single judge is unreliable and even a panel...

By Mufeng Yang, Junwei Yu, Yepeng Ding

llms agents benchmarks

arXiv AI

Jun 2

Demystifying Multi-Agent Debate: The Role of Confidence and Diversity

arXiv:2601. 19921v2 Announce Type: replace-cross Abstract: Multi-agent debate (MAD) is widely used to improve large language model (LLM) performance through test-time scaling, yet recent work shows that vanilla MAD often underperforms simple majority vote despite higher computational cost.

By Xiaochen Zhu, Caiqi Zhang, Yizhou Chi, Tom Stafford, Nigel Collier, Andreas Vlachos

llms agents benchmarks

Related stories