arXiv Computation and Language

Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments

arXiv Computation and Language
Aug 27

Reasoning or Rambling? Exploring the Effect of Thinking on Agent Persuasion

The paper investigates how explicit reasoning in Large Reasoning Models (LRMs) affects their ability to persuade and be persuaded. Experiments on objective and subjective tasks reveal a Persuasion Duality: reasoning boosts an agent’s persuasive power by about 21 percentage points while also making it less susceptible to incorrect persuasion by up to 10 percentage points. However, the study finds that persuasiveness often relies on superficial cues like response length and repetition rather than logical validity, and that persuasion can amplify or attenuate non‑linearly across multi‑hop agent chains. The authors also propose an attention‑guided prompt‑level adversarial argument detection method that improves agent robustness.

By Haodong Zhao, Jidong Li, Zhaomin Wu, Tianjie Ju, Zhuosheng Zhang, Bingsheng He, Gongshen Liu
arXiv Computation and Language
Aug 27

Belief Cascades Drive Persuasion in LLM Agent Networks

The paper introduces a controlled testbed to study how goal‑directed persuaders shift stances in networks of large language model agents, using real‑world ego‑network topologies. Experiments across four LLM backbones, five graph structures, and 55 policy statements show that persuasion dynamics depend on topology, competition, topic, and model prior. The study finds that direct exposure predicts stance change, peer relays have measurable influence, and that post‑text analysis alone misses important movement, highlighting the need to evaluate multi‑agent persuasion through trajectory‑level processes, belief probes, exposure provenance, and action logs.

By Haoyi Qiu, Genglin Liu, Pranav Narayanan Venkit, Kung-Hsiang Huang, Saadia Gabriel, Chien-Sheng Wu, Nanyun Peng
Hugging Face Trending Papers
Aug 12

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior.

arXiv Computation and Language
1d ago

Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement

The paper introduces a novel framework for assessing second‑order bias in large language models (LLMs), defined as bias in how an LLM judges the acceptability of biased content. Using principles from entitlement epistemology, the authors design a reasoning task that asks LLMs to determine whether a biased text is acceptable for specific demographic groups, and propose two metrics to quantify biased judgments. Experiments on both open‑source and closed‑source models reveal that the task bypasses safety guardrails, uncovers systematic variations across target groups, and demonstrates that models still rely on demographic labels when evaluating bias.

By Ramaravind Kommiya Mothilal, Terry Jingchen Zhang, Raiyan Ahmed, Zhijing Jin, Shion Guha, Syed Ishtiaque Ahmed
arXiv Computation and Language
2d ago

Evaluating the Capabilities of LLMs for Persuasive Dialogue

The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.

By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn