arXiv AI

HauntAttack: When Attack Follows Reasoning as a Shadow

arXiv:2506. 07031v5 Announce Type: replace-cross Abstract: Emerging Large Reasoning Models (LRMs) consistently excel in mathematical and reasoning tasks, showcasing remarkable capabilities.

arXiv AI
Sep 24

Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposure

Large Reasoning Models (LRMs) use explicit chain‑of‑thought reasoning and large context windows to perform complex tasks, but these features create new attack surfaces. The paper introduces SRCF, an attack that steers LRMs by prepending counter‑aligned few‑shot conversations with explicit CoT traces, causing unsafe outputs on harmful queries and unwarranted refusals on benign ones, without needing model internals. To counter this, the authors propose ARCF, a post‑training defense that exposes models to counter‑aligned conversational contexts while enforcing aligned targets, improving safety and helpfulness without harming utility.

By Xiangyu Zhou, Saleh Zare Zade, Dongxiao Zhu
arXiv Machine Learning
Sep 21

OverThink: Slowdown Attacks on Reasoning LLMs

The paper introduces OverThink, a slowdown attack that forces reasoning language models (RLMs) to produce many more reasoning tokens while still giving correct answers. By injecting decoy reasoning problems—such as Markov decision processes, language translation, or graphic comprehension—into the model’s context, attackers can dramatically increase token generation (up to 46× on SQuAD and 17× on coding agents). The study evaluates the attack on both proprietary and open-source RLMs across multiple datasets, explores multimodal and coding‑agent variants, and tests several defenses, concluding that defending against OverThink is challenging and that newer RLMs are even more vulnerable due to higher per‑token costs and increased reasoning token usage.

By Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, Eugene Bagdasarian
arXiv AI
Sep 2

Does Reasoning Mitigate Backdoor Attacks? A Neuro-Symbolic Perspective

The paper investigates whether neuro‑symbolic (NeSy) AI models, which combine neural perception with symbolic reasoning, can mitigate backdoor attacks. It presents a systematic evaluation comparing the NeSy framework DeepProbLog to baseline neural networks across eight backdoor settings and four reasoning tasks. Results indicate that NeSy models are generally more robust than pure neural models, but their resilience depends heavily on how strictly the reasoning process is enforced and its alignment with the attack target.

By Marco Antonio Corallo, Andrea Agiollo, Mauro Conti, Alberto Giaretta
arXiv AI
Aug 26

TRACE: An Evidence-Grounded Benchmark for Safety Evaluation of Large Reasoning Models

TRACE is a new benchmark that evaluates the safety of Large Reasoning Models (LRMs) across the entire inference pipeline, including prompts, reasoning traces, and final responses. It provides prompts in two languages covering nine risk categories and ten attack strategies, and for each prompt four LRMs generate traces and responses that are annotated for safety with supporting evidence extracted from the source text. Evaluation of 18 guardrail models on TRACE shows that detecting unsafe content in reasoning traces is much harder than in prompts or final responses, and that current models struggle to extract the necessary evidence.

By Zhenyu Wu, Siyuan Chen, Changchun Yang, Jiaqi Dong, Min Zhou, Ali Almadan, Talal Hammad, Faisal Wahbo, Aminullah Tora, Mona Alshahrani, Xin Gao
arXiv AI
Sep 17

First Token Matters: Understanding Safety Collapse in Large Reasoning Models

The paper investigates why large reasoning models (LRMs) lose safety alignment when faced with harmful queries. By analyzing token-level refusal dynamics, the authors identify a vulnerability called Onset Refusal Collapse (ORC), where the refusal signal drops sharply at the first generated token, leading to unsafe responses. They introduce SafeToken, a lightweight inference-time intervention that injects a learned safety anchor at reasoning onset, which mitigates ORC, improves safety on harmful-query benchmarks, and largely preserves reasoning utility.

By Yizheng Yang, Haining Yu, Yuechen Wang, Yikai Hou, Xing Fu, Jinbo Yang, Tianqing Zhu