arXiv Computation and Language

Benchmarking Argumentative Behaviour of LLMs: A Study of Defences Against Character Attacks

The paper investigates how well large language models (LLMs) can handle character attacks—ad hominem arguments—in political debates. By analyzing natural political dialogues and comparing LLM-generated responses to a corpus of U.S. presidential debates, the study finds that most LLMs favor logical defenses and rarely use ethos-based counterattacks. The authors suggest that safety fine‑tuning limits LLMs’ strategic options, preventing them from fully engaging in realistic political discourse.

arXiv Computation and Language

Sep 1

Evaluating the Capabilities of LLMs for Persuasive Dialogue

The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.

By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn

llms rag agents

arXiv AI

Jun 4

PoliticsBench: Benchmarking Political Values in Large Language Models with Multi-Turn Roleplay

arXiv:2603. 23841v2 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) are increasingly used as primary sources of information, their potential for political bias may impact their objectivity.

By Rohan Khetan, Ashna Khetan

llms benchmarks safety

arXiv Computation and Language

Sep 15

A Multi-Stage Agentic Framework for Effective Counter-Narrative Generation and Refinement

arXiv:2609.14178v1 Announce Type: new Abstract: The rapid diffusion of hate speech and misinformation on social networks challenges democratic societies, since direct suppression efforts may deepen p...

By Carmel Kronfeld, Sharva Gogawale, Tetsuro Kobayashi, Irad Ben-Gal

llms agents diffusion safety

arXiv Computation and Language

Sep 10

A Resource for Enthymeme Detection in Controversial Political Discourse

arXiv:2606.12186v2 Announce Type: replace Abstract: Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjec...

By Martial Pastor, Nelleke Oostdijk

nlp

Hugging Face Trending Papers

Jun 10

A Resource for Enthymeme Detection in Controversial Political Discourse

Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjective. We present a resource of 1,482 tweets from politically controversial discourse, annotated by five annotators for the presence of enthymemes and their argument structure, designed to study label variation.

nlp

arXiv AI

Aug 18

Inference-Time Mitigation of Adversarial Political Bias in Large Language Models

arXiv:2608. 14629v1 Announce Type: cross Abstract: As Large Language Models (LLMs) become the mainstay for information retrieval and summarization tasks, ensuring that they are always non-partisan and invulnerable to political bias is a critical step towards safer and more trustworthy Artificial Intelligence (AI).

By Tejaswi V. Panchagnula, Bruce Coburn, Bryce J. Dietrich, Robert X. Browning, Edward J. Delp, Fengqing Zhu

llms nlp reinforcement-learning fine-tuning safety

arXiv AI

Jun 9

"I understand your perspective": LLM Persuasion and Sycophancy through the Lens of Communicative Action Theory

arXiv:2606. 08076v1 Announce Type: cross Abstract: Large Language Models (LLMs) can generate high-quality arguments, yet their ability to engage in nuanced and persuasive communicative actions remains largely unexplored.

By Esra D\"onmez, Agnieszka Falenska

llms

arXiv AI

Aug 13

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

arXiv:2608. 11624v1 Announce Type: cross Abstract: Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions.

By Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur, Dilek Hakkani-T\"ur

llms agents reinforcement-learning safety

arXiv AI

Jul 22

"I understand your perspective": LLM Persuasion through the Lens of Communicative Action Theory

arXiv:2606. 08076v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can generate high-quality arguments, yet their ability to engage in nuanced and persuasive communicative actions remains largely unexplored.

By Esra D\"onmez, Agnieszka Falenska

llms

arXiv AI

Sep 1

How Identity and Opinion Shape Political Sycophancy in LLMs

arXiv:2608.29198v1 Announce Type: new Abstract: As Large Language Models (LLMs) increasingly encourage users to disclose personal profiles for tailored assistance, measuring their political alignment...

By Li-Ni Fu, Chang-Chih Meng, Chien-Hua Chen, Hen-Hsen Huang, I-Chen Wu

llms benchmarks safety

Hugging Face Trending Papers

Aug 12

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior.

llms agents reinforcement-learning safety

arXiv AI

Jul 24

Robust Critics: Defending LLMs Against Multi-Turn Attacks

arXiv:2607. 20472v1 Announce Type: new Abstract: When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question?

By Roman Belaire, Arunesh Sinha, Pradeep Varakantham

llms reinforcement-learning fine-tuning safety

Related stories