arXiv Computation and Language September 1, 2026 By Ruoxuan Li, Pinqiao Wang, Sheng Li, Cameron Robert Jones

You Shouldn't Have Asked: A Pragmatics-Inspired Taxonomy for Evaluating LLM Refusals

Read the original on arXiv Computation and Language →

The paper introduces a pragmatics-inspired taxonomy for evaluating how large language models (LLMs) refuse unsafe or inappropriate requests. By applying this framework to 16 modern LLMs across 14 harm categories, the authors find that while refusals are generally explicit and morally charged, they often lack interpersonal facework and instead offer safer alternatives, which can be problematic in sensitive contexts. The study argues for alignment evaluations that assess not just whether LLMs refuse, but how they do so in a contextually adaptive and socially responsible manner.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Aug 25

Hidden in the Request: Explaining Unethical LLM Compliance through Token Relevance

arXiv:2608.23264v1 Announce Type: new Abstract: Although Large Language Models (LLMs) are aligned to optimize for both helpfulness and harmlessness, these dual objectives may conflict, inevitably lea...

By Or Biton, Tomer Krichli, Itai Allouche, Joseph Keshet

llms safety

arXiv AI

Jul 9

Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas

arXiv:2601. 21433v2 Announce Type: replace Abstract: Language models are increasingly consulted on ethically consequential questions, yet the stance a model expresses may not survive a change in framing.

By Katherine Elkins, Jon Chun

llms safety

arXiv AI

Jun 26

Reducing Conversational Escalation in Large Language Model Dialogue with Nonviolent Communication Constraints

arXiv:2606. 26106v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in emotionally charged situations involving interpersonal conflict, frustration, and distress.

By Zhixing Sun, Shenghe Xu, Tao Li

llms agents safety

arXiv Computation and Language

Aug 28

DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs

DeflectBench is a new benchmark that evaluates how large language models (LLMs) generate rhetorical fallacies when prompted. The study tests 23,990 generations from four leading models using three deflection strategies (whataboutism, ad hominem, red herring), seven prompt framings, and 80 claims across four controversy levels. Results show that refusal to produce fallacies depends mainly on request structure, with prompt framing and fallacy type dramatically affecting compliance rates.

By Art Kanke

llms robotics benchmarks safety

arXiv AI

Jul 29

Do Models Fake Alignment Without Clear Consequences?

arXiv:2607. 24758v1 Announce Type: new Abstract: Large language models are capable of recognizing evaluation contexts and altering their behavior to reflect evaluator expectations rather than typical deployment behaviors, a phenomenon known as alignment faking.

By Cole Alexander Niblett, Alexander Chabot Nanni, Anita K. Rao

llms agents safety

arXiv Machine Learning

Jun 2

Investigating and Alleviating Harm Amplification in LLM Interactions

arXiv:2606. 02423v1 Announce Type: cross Abstract: Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactions.

By Ruohao Guo, Wei Xu, Alan Ritter

llms benchmarks safety

One email a morning, machine-written

Related stories

Hidden in the Request: Explaining Unethical LLM Compliance through Token Relevance

Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas

Reducing Conversational Escalation in Large Language Model Dialogue with Nonviolent Communication Constraints

DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs

Do Models Fake Alignment Without Clear Consequences?

Investigating and Alleviating Harm Amplification in LLM Interactions