arXiv AI By Xiang Yin, Adam Dejl, Antonio Rago, Lihu Chen, Francesca Toni

A Theory of Post-hoc Debate Judgement

Read the original on arXiv AI →

The paper proposes a theory for judging post-hoc debates in AI, focusing on properties like reproducibility, robustness, groundedness, and explainability. It evaluates two debate‑judgement methods—LLM judges and formal computational argumentation semantics—finding similar accuracy but noting that argumentation semantics offers stronger formal guarantees. The study suggests that argumentation semantics is a preferable framework for principled debate judges in AI systems.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 28

OmouAI: Argumentative Human-AI Policy Deliberation with Simulated Personas

OmouAI is an interactive deliberation system that combines large language models with computational argumentation to facilitate policy debates involving humans and simulated personas such as stakeholders, experts, or devil’s advocates. Each persona generates its own arguments, which are assembled into a shared argumentation framework that users can contest, add to, or revise, ensuring human oversight. The system evaluates arguments using deterministic argumentative semantics against external goals like the UN Sustainable Development Goals, providing faithful explanations and indicating how policy recommendations affect those goals.

By Stylianos Loukas Vasileiou, Antonio Rago, William Yeoh, Georgina Curto
arXiv AI
Jul 23

Avoiding Obfuscation with Prover-Estimator Debate

arXiv:2506. 13609v2 Announce Type: replace Abstract: Training powerful AI systems to exhibit desired behaviors hinges on the ability to provide accurate human supervision on increasingly complex tasks.

By Jonah Brown-Cohen, Geoffrey Irving, Georgios Piliouras, Lijie Chen, Jiawei Li, Zhiyang Xun
arXiv AI
Oct 5

How to Have a Sensitive Debate: An Instance-Optimal Protocol for AI Debate

The paper introduces a new AI debate protocol that improves upon previous designs for problems with stable decompositions into subproblems. It guarantees correctness in a worst‑case sense, establishes honesty and correctness as a dominant‑strategy equilibrium for both debaters, and proves instance‑wise optimality through black‑box lower bounds. The authors connect stable problem decompositions to fractional block sensitivity from query complexity to achieve these results.

By Jiawei Li, Zhiyang Xun, Lijie Chen, Jonah Brown-Cohen
arXiv Computation and Language
Sep 1

Evaluating the Capabilities of LLMs for Persuasive Dialogue

The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.

By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn