arXiv AI
SAVeR$^2$: Reasoning-based Safety Alignment for Large Reasoning Models via Verifiable Rewards
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 30
Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models
arXiv:2609.36254v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) are commonly trained with reinforcement learning (RL) to improve their generation of chain-of-thought (CoT) reasoning bef...
arXiv AI
Sep 15
Beyond Safe Answers: Segment-Aware Listwise Alignment for Reasoning Safety in Large Reasoning Models
The paper introduces SaLT-DPO, a Segment‑aware Listwise Target DPO method designed to improve safety in Large Reasoning Models (LRMs). It decomposes responses into reasoning and answer segments, scores each segment’s safety, and aligns length‑normalized rewards with soft target distributions over multiple candidates. The approach also incorporates joint safety coherence regularization and utility anchoring to reduce unsafe content while maintaining compliance and reasoning performance.
arXiv AI
Oct 19
Rethinking GRPO for Large Reasoning Model Safety Alignment
The paper "Rethinking GRPO for Large Reasoning Model Safety Alignment" identifies two failure modes in Group Relative Policy Optimization—advantage collapse and unsafe diversification—that weaken safety learning in large reasoning models. It proposes the Beacon framework, which includes Safety Signal Recovery (SSR) and Safety-Utility Decoupled Alignment (SUDA), to restore effective safety signals and mitigate over-refusal. Experiments on Qwen3 and DeepSeek-R1-Distill models across eight benchmarks demonstrate that Beacon improves safety performance by up to 32.83 percentage points while keeping over-refusal low.
arXiv AI
Aug 11
Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs
arXiv:2608. 09542v1 Announce Type: cross Abstract: Large reasoning models (LRMs) achieve remarkable success on complex tasks but remain vulnerable to harmful prompts that induce unsafe outputs.
arXiv AI
Oct 12
ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models
ReasoningShield introduces a benchmark and lightweight models for detecting unsafe content in the intermediate reasoning steps of Large Reasoning Models (LRMs). The benchmark contains 9.2K annotated (Query, CoT) pairs across 10 risk categories, 3 safety levels, and 8 LRMs, and the proposed models achieve 91.8% F1, outperforming existing moderation tools by significant margins. The study highlights that current moderation tools degrade by up to 35.2% on Chain-of-Thought reasoning compared to traditional answer moderation.
arXiv AI
Sep 24
Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposure
Large Reasoning Models (LRMs) use explicit chain‑of‑thought reasoning and large context windows to perform complex tasks, but these features create new attack surfaces. The paper introduces SRCF, an attack that steers LRMs by prepending counter‑aligned few‑shot conversations with explicit CoT traces, causing unsafe outputs on harmful queries and unwarranted refusals on benign ones, without needing model internals. To counter this, the authors propose ARCF, a post‑training defense that exposes models to counter‑aligned conversational contexts while enforcing aligned targets, improving safety and helpfulness without harming utility.