arXiv Computation and Language September 16, 2026 By Zhuoang Cai

Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion

Read the original on arXiv Computation and Language →

The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Hugging Face Trending Papers

Oct 26

The Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red Teaming

Automatic red teaming increasingly relies on large language models (LLMs) to iteratively diagnose failed attacks and propose improved adversarial prompts or injections. Although prior work strengthens...

llms agents multimodal safety

arXiv AI

Aug 13

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

arXiv:2608. 11624v1 Announce Type: cross Abstract: Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions.

By Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur, Dilek Hakkani-T\"ur

llms agents reinforcement-learning safety

arXiv AI

Jul 24

Robust Critics: Defending LLMs Against Multi-Turn Attacks

arXiv:2607. 20472v1 Announce Type: new Abstract: When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question?

By Roman Belaire, Arunesh Sinha, Pradeep Varakantham

llms reinforcement-learning fine-tuning safety

arXiv AI

6d ago

The Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red Teaming

arXiv:2610.34965v1 Announce Type: cross Abstract: Automatic red teaming increasingly relies on large language models (LLMs) to iteratively diagnose failed attacks and propose improved adversarial pro...

By Mintong Kang, Stephen Rawls, Melissa Kazemi Rad, Jian Ni, Bo Li

llms agents multimodal safety

Hugging Face Trending Papers

Aug 12

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior.

llms agents reinforcement-learning safety

arXiv AI

Jul 22

A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

arXiv:2606. 25476v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated remarkable performance across natural language processing tasks, yet their deployment in high-stakes applications raises critical concerns regarding reliability, safety, and trustworthiness.

By Abrar Alotaibi, Raed Mughus, Moataz Ahmed

llms nlp safety

One email a morning, machine-written

Related stories

The Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red Teaming

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Robust Critics: Defending LLMs Against Multi-Turn Attacks

The Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red Teaming

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation