arXiv Computation and Language

Evaluating the Capabilities of LLMs for Persuasive Dialogue

The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.

arXiv Computation and Language
Sep 25

Benchmarking Argumentative Behaviour of LLMs: A Study of Defences Against Character Attacks

The paper investigates how well large language models (LLMs) can handle character attacks—ad hominem arguments—in political debates. By analyzing natural political dialogues and comparing LLM-generated responses to a corpus of U.S. presidential debates, the study finds that most LLMs favor logical defenses and rarely use ethos-based counterattacks. The authors suggest that safety fine‑tuning limits LLMs’ strategic options, preventing them from fully engaging in realistic political discourse.

By Ewelina Gajewska, Katarzyna Budzynska, Jaroslaw Chudziak
Hugging Face Trending Papers
Jun 10

A Resource for Enthymeme Detection in Controversial Political Discourse

Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjective. We present a resource of 1,482 tweets from politically controversial discourse, annotated by five annotators for the presence of enthymemes and their argument structure, designed to study label variation.

arXiv Computation and Language
Aug 24

ARGUS: Theory-of-Mind Guided Argument Generation with Strategy-Aware Planning and Knowledge Grounding

ARGUS is a new agent-based framework for persuasive argument generation that incorporates a Theory-of-Mind Reasoner to model audience beliefs and values. It uses a component-aware planner to break arguments into subtopics, assign rhetorical functions (logos, pathos, ethos, kairos), and guide evidence retrieval during planning. A refinement module iteratively addresses multi-dimensional weaknesses, and evaluations on three benchmarks show ARGUS outperforming strong baselines and effectively shifting resistant audience stances.

By Zhe Hu
arXiv Machine Learning
Aug 4

OpenDebateEvidence: A Massive-Scale Argument Mining and Summarization Dataset

arXiv:2406. 14657v4 Announce Type: replace-cross Abstract: We introduce OpenDebateEvidence, a comprehensive dataset for argument mining and summarization sourced from the American Competitive Debate community.

By Allen Roush, Yusuf Shabazz, Arvind Balaji, Peter Zhang, Stefano Mezza, Markus Zhang, Sanjay Basu, Sriram Vishwanath, Mehdi Fatemi, Ravid Shwartz-Ziv
arXiv Computation and Language
Aug 31

Persuasion Index: A Theory-Guided Framework for Persuasion Analysis

The paper introduces the Persuasion Index (PI), a taxonomy of 15 persuasion dimensions grounded in psychological and communication theories, implemented with 55 lexicon- and rule-based sub-features. PI is modular, allowing individual detectors to be swapped while preserving its theoretical framework. Evaluations on four English argumentative datasets show that PI provides a shared, lightweight feature space that captures meaningful predictive signals and reveals consistent dimension-level associations with persuasion outcomes, with variations across topics and stances.

By Liancheng Gong, Zhiyang Wang, Yiwei Xu, Julia Mendelsohn
arXiv AI
Aug 20

A Theory of Post-hoc Debate Judgement

The paper proposes a theory for judging post-hoc debates in AI, focusing on properties like reproducibility, robustness, groundedness, and explainability. It evaluates two debate‑judgement methods—LLM judges and formal computational argumentation semantics—finding similar accuracy but noting that argumentation semantics offers stronger formal guarantees. The study suggests that argumentation semantics is a preferable framework for principled debate judges in AI systems.

By Xiang Yin, Adam Dejl, Antonio Rago, Lihu Chen, Francesca Toni