arXiv Computation and Language September 16, 2026 By Zhuoang Cai

Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion

Read the original on arXiv Computation and Language →

The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Aug 13

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

arXiv:2608. 11624v1 Announce Type: cross Abstract: Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions.

By Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur, Dilek Hakkani-T\"ur

llms agents reinforcement-learning safety

arXiv AI

Jul 24

Robust Critics: Defending LLMs Against Multi-Turn Attacks

arXiv:2607. 20472v1 Announce Type: new Abstract: When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question?

By Roman Belaire, Arunesh Sinha, Pradeep Varakantham

llms reinforcement-learning fine-tuning safety

Hugging Face Trending Papers

Aug 12

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior.

llms agents reinforcement-learning safety

arXiv AI

Jul 22

A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

arXiv:2606. 25476v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated remarkable performance across natural language processing tasks, yet their deployment in high-stakes applications raises critical concerns regarding reliability, safety, and trustworthiness.

By Abrar Alotaibi, Raed Mughus, Moataz Ahmed

llms nlp safety

arXiv Machine Learning

Jul 30

ToxScreen: Detecting Whether an LLM Has Been Poisoned

arXiv:2607. 26849v1 Announce Type: cross Abstract: As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time.

By Anthony Hughes, Nicole Xing, Collin Francel, Andy Kim, Andrew Draganov

llms benchmarks

arXiv AI

Jul 24

Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind

arXiv:2604. 11666v2 Announce Type: replace-cross Abstract: As large language models (LLMs) become the engine behind conversational systems, their ability to reason about the intentions and states of their dialogue partners (i.

By Hanqi Xiao, Vaidehi Patil, Zaid Khan, Hyunji Lee, Elias Stengel-Eskin, Mohit Bansal

llms agents reinforcement-learning safety

One email a morning, machine-written

Related stories

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Robust Critics: Defending LLMs Against Multi-Turn Attacks

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

ToxScreen: Detecting Whether an LLM Has Been Poisoned

Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind