arXiv AI
Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 31
Large Reasoning Models Learn Better Alignment from Flawed Thinking
Large reasoning models generate chain-of-thought (CoT) before answering but struggle with safety alignment and can be misled by flawed premises. The paper introduces RECAP, a reinforcement learning approach that trains models to override flawed reasoning paths and produce safe, helpful responses without extra training cost. RECAP improves safety, jailbreak robustness, and reduces overrefusal while preserving core reasoning abilities and inference token budget.
arXiv Machine Learning
Jul 1
Addressing Over-Refusal in LLMs with Competing Rewards
arXiv:2606. 31748v1 Announce Type: new Abstract: Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones.
arXiv AI
Sep 17
Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
The paper introduces "cunning questions"—non‑safety prompts that contain misleading premises or subtle inconsistencies—to train large language models (LLMs) to scrutinize underlying intent and assumptions. Experiments show that incorporating these questions improves robustness against out‑of‑distribution jailbreak attacks and enhances subsequent safety fine‑tuning, achieving a new state‑of‑the‑art reduction in mean ASR from 17.40% to 15.05% across nine backbone–benchmark combinations. The authors argue that this training fosters vigilance, enabling models to prioritize safety judgments before engaging in harmful planning.
arXiv AI
Jun 8
Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry
arXiv:2603. 26846v2 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) expand in capability and application scope, their trustworthiness becomes critical.
arXiv AI
Sep 15
Beyond Safe Answers: Segment-Aware Listwise Alignment for Reasoning Safety in Large Reasoning Models
The paper introduces SaLT-DPO, a Segment‑aware Listwise Target DPO method designed to improve safety in Large Reasoning Models (LRMs). It decomposes responses into reasoning and answer segments, scores each segment’s safety, and aligns length‑normalized rewards with soft target distributions over multiple candidates. The approach also incorporates joint safety coherence regularization and utility anchoring to reduce unsafe content while maintaining compliance and reasoning performance.
arXiv AI
Jul 7
Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models
arXiv:2607. 02914v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse applications, yet ensuring their simultaneous safety, helpfulness, and trustworthiness remains a persistent challenge.