arXiv Computation and Language August 28, 2026 By Art Kanke

DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs

Read the original on arXiv Computation and Language →

DeflectBench is a new benchmark that evaluates how large language models (LLMs) generate rhetorical fallacies when prompted. The study tests 23,990 generations from four leading models using three deflection strategies (whataboutism, ad hominem, red herring), seven prompt framings, and 80 claims across four controversy levels. Results show that refusal to produce fallacies depends mainly on request structure, with prompt framing and fallacy type dramatically affecting compliance rates.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Aug 18

Characterizing Rhetorical Misalignment in Decision-Making with Language Models

arXiv:2608. 14630v1 Announce Type: cross Abstract: Human decision-making is often shaped by a range of well-documented cognitive biases.

By Zirui Cheng, Joey Chan, Simo Du, Chenhao Tan, Yue Guo, Hao Peng

llms safety

arXiv AI

Jul 9

Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas

arXiv:2601. 21433v2 Announce Type: replace Abstract: Language models are increasingly consulted on ethically consequential questions, yet the stance a model expresses may not survive a change in framing.

By Katherine Elkins, Jon Chun

llms safety

arXiv AI

Jun 26

ConflictScore: Identifying and Measuring How Language Models Handle Conflicting Evidence

arXiv:2606. 26437v1 Announce Type: cross Abstract: Existing metrics for factuality and faithfulness evaluate whether an answer is supported or contradicted by its grounding documents, but they fail to capture when both supporting and contradicting evidence coexist.

By Siyi Liu, Aaron Halfaker, Dan Roth, Patrick Xia

llms benchmarks

arXiv AI

Aug 13

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs

arXiv:2608. 11415v1 Announce Type: cross Abstract: Large language models are being proposed as agents in scientific workflows, in domains where no downstream verifier exists.

By Valentin Rodionov, Shamil Assylbekov

llms agents benchmarks safety

arXiv AI

4d ago

The Invisible Editorial Layer: Formalizing Undisclosed Inference-Time Steering, Probability Placement, and the Attribution Problem in Deployed Language Models

The paper argues that modern inference pipelines add an unseen layer of control between a language model’s frozen weights and its output, altering probability distributions before token selection. It introduces the concepts of the Inference Attribution Problem, Probability Placement, and Inference Policy Transparency to describe how such interventions can bias generated language toward specific frames and how these biases cannot be traced solely to model weights. The authors discuss the governance, security, and economic implications of these undisclosed inference policies, referencing EU AI Act, Digital Services Act, and FTC doctrines.

By Augusto Camargo

llms safety

arXiv Computation and Language

5d ago

Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models

arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...

By Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Li Chen

llms fine-tuning safety

One email a morning, machine-written

Related stories

Characterizing Rhetorical Misalignment in Decision-Making with Language Models

Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas

ConflictScore: Identifying and Measuring How Language Models Handle Conflicting Evidence

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs

The Invisible Editorial Layer: Formalizing Undisclosed Inference-Time Steering, Probability Placement, and the Attribution Problem in Deployed Language Models

Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models