arXiv AI

A Theory of Post-hoc Debate Judgement

The paper proposes a theory for judging post-hoc debates in AI, focusing on properties like reproducibility, robustness, groundedness, and explainability. It evaluates two debate‑judgement methods—LLM judges and formal computational argumentation semantics—finding similar accuracy but noting that argumentation semantics offers stronger formal guarantees. The study suggests that argumentation semantics is a preferable framework for principled debate judges in AI systems.

arXiv AI
Sep 28

OmouAI: Argumentative Human-AI Policy Deliberation with Simulated Personas

OmouAI is an interactive deliberation system that combines large language models with computational argumentation to facilitate policy debates involving humans and simulated personas such as stakeholders, experts, or devil’s advocates. Each persona generates its own arguments, which are assembled into a shared argumentation framework that users can contest, add to, or revise, ensuring human oversight. The system evaluates arguments using deterministic argumentative semantics against external goals like the UN Sustainable Development Goals, providing faithful explanations and indicating how policy recommendations affect those goals.

By Stylianos Loukas Vasileiou, Antonio Rago, William Yeoh, Georgina Curto
arXiv AI
Jul 23

Avoiding Obfuscation with Prover-Estimator Debate

arXiv:2506. 13609v2 Announce Type: replace Abstract: Training powerful AI systems to exhibit desired behaviors hinges on the ability to provide accurate human supervision on increasingly complex tasks.

By Jonah Brown-Cohen, Geoffrey Irving, Georgios Piliouras, Lijie Chen, Jiawei Li, Zhiyang Xun
arXiv AI
Oct 5

How to Have a Sensitive Debate: An Instance-Optimal Protocol for AI Debate

The paper introduces a new AI debate protocol that improves upon previous designs for problems with stable decompositions into subproblems. It guarantees correctness in a worst‑case sense, establishes honesty and correctness as a dominant‑strategy equilibrium for both debaters, and proves instance‑wise optimality through black‑box lower bounds. The authors connect stable problem decompositions to fractional block sensitivity from query complexity to achieve these results.

By Jiawei Li, Zhiyang Xun, Lijie Chen, Jonah Brown-Cohen
arXiv Computation and Language
Sep 1

Evaluating the Capabilities of LLMs for Persuasive Dialogue

The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.

By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn
arXiv AI
Sep 28

Towards Mitigating Fabricated Consensus: The Active Provenance Gate for Multi-Agent Debate Synthesis

The paper introduces the Active Provenance Gate (APG), a post‑debate verification layer for multi‑agent debate synthesis that audits debate logs, applies self‑correction, and blocks unsupported claims before publication. Empirical studies show that APG more than doubles provenance fidelity in crisis simulations and that users prefer explicit failure reports over fabricated consensus. The work shifts data origin tracing from passive logging to active conditional blocking, addressing safety gaps in large‑language‑model‑based debate systems.

By Jakub Mas{\l}owski, Jaros{\l}aw A. Chudziak