← Back to all news
arXiv AI By Jinghao Pang, Jitai Hao, Qiang Huang, Zhaochun Ren, Jun Yu

Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment

Read the original on arXiv AI →

The paper introduces SSRFT (Supervised Safe-Role Fine‑Tuning), a new framework that reframes safety alignment for large language models as internalizing a predefined safe role rather than relying on explicit refusal patterns. SSRFT builds a Safe‑Role Question‑Answer dataset from psychometric questions, limited jailbreak prompts, and a safe‑role description, then synthesizes and validates role‑consistent responses to expose models to diverse safety scenarios. Experiments on multiple base and instruct models demonstrate that SSRFT yields more robust and generalizable safety alignment, better resilience to prefilling attacks, improved generalization to unseen jailbreak domains, and reduced over‑refusal on benign queries while preserving general capabilities.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jun 25

RAS: Measuring LLM Safety Through Refusal Alignment

arXiv:2606. 25750v1 Announce Type: cross Abstract: Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy.

By Chang-Chieh Huang, Yan-Lun Chen, Chia-Mu Yu, Wei-Bin Lee
llmssafety
More like this →
arXiv AI
Sep 17

Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models

The paper introduces "cunning questions"—non‑safety prompts that contain misleading premises or subtle inconsistencies—to train large language models (LLMs) to scrutinize underlying intent and assumptions. Experiments show that incorporating these questions improves robustness against out‑of‑distribution jailbreak attacks and enhances subsequent safety fine‑tuning, achieving a new state‑of‑the‑art reduction in mean ASR from 17.40% to 15.05% across nine backbone–benchmark combinations. The authors argue that this training fosters vigilance, enabling models to prioritize safety judgments before engaging in harmful planning.

By Youjia Wang, Lin Xu, Yang Sun, Yuxiao Lu, Chengfang Fang, Jie Shi
llmsfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Sep 7

The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs

The paper investigates a continuation-triggered jailbreak in large language models, showing that moving an instruction suffix can markedly boost jailbreak success. By performing mechanistic interpretability at the attention‑head level, the authors reveal that the jailbreak arises from a competition between the model’s natural continuation drive and safety defenses learned during alignment. They introduce Head Competition Steering (HCS), an inference‑time technique that exploits this competition to suppress harmful outputs and distill the approach into a student model for efficient safety improvements.

By Yonghong Deng, Zhen Yang, Ping Jian, Xinyue Zhang, Zhongbin Guo, Chengzhi Li, Junxi Yin
llmsefficiencysafety
More like this →