arXiv AI By Xiangyu Zhou, Saleh Zare Zade, Dongxiao Zhu

Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposure

Read the original on arXiv AI →

Large Reasoning Models (LRMs) use explicit chain‑of‑thought reasoning and large context windows to perform complex tasks, but these features create new attack surfaces. The paper introduces SRCF, an attack that steers LRMs by prepending counter‑aligned few‑shot conversations with explicit CoT traces, causing unsafe outputs on harmful queries and unwarranted refusals on benign ones, without needing model internals. To counter this, the authors propose ARCF, a post‑training defense that exposes models to counter‑aligned conversational contexts while enforcing aligned targets, improving safety and helpfulness without harming utility.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Sep 21

OverThink: Slowdown Attacks on Reasoning LLMs

The paper introduces OverThink, a slowdown attack that forces reasoning language models (RLMs) to produce many more reasoning tokens while still giving correct answers. By injecting decoy reasoning problems—such as Markov decision processes, language translation, or graphic comprehension—into the model’s context, attackers can dramatically increase token generation (up to 46× on SQuAD and 17× on coding agents). The study evaluates the attack on both proprietary and open-source RLMs across multiple datasets, explores multimodal and coding‑agent variants, and tests several defenses, concluding that defending against OverThink is challenging and that newer RLMs are even more vulnerable due to higher per‑token costs and increased reasoning token usage.

By Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, Eugene Bagdasarian
arXiv AI
Nov 6

SAVeR$^2$: Reasoning-based Safety Alignment for Large Reasoning Models via Verifiable Rewards

SAVeR$^2$ is a safety alignment method for Large Reasoning Models that decomposes safety objectives into four fine‑grained verifiable reward signals and uses a difficulty‑aware data selection strategy to stabilize training. It introduces a reasoning‑preserving gradient projection mechanism that resolves conflicts between safety and reasoning gradients, eliminating the safety tax. Experiments on models from 1.5B to 671B show that SAVeR$^2$ improves safety performance and reduces over‑refusal rates without harming general reasoning abilities.

By Yichen Sun, Qirui Liu, Yan Wang, Jianan Lin, Linbo Jiang, Shiyu Wang, Zhibo Wang, Zhixuan Chu