The paper investigates how large language models balance helpfulness and safety by refusing harmful queries while responding to benign ones. It decomposes safety-tuning responses into a boilerplate refusal statement and a rationale, finding that the statement causes false refusals by relying on superficial cues. Training on rationales alone reduces false refusals without compromising safety performance, suggesting that fine‑grained safety supervision is essential for better alignment.
By Minji Kim, Hyounghun Kim
The paper investigates how different post‑training methods—supervised fine‑tuning, reasoning‑augmented fine‑tuning, and preference optimization (ORPO)—affect the internal computation of refusal behavior in language models. Experiments on Llama‑3.1‑8B, Gemma‑2‑9B, and Qwen3‑8B show that reasoning‑augmented training consistently creates a distinct refusal computation across models, while the architecture influences the internal structure and steerability of refusal. None of the studied methods simultaneously achieve a distributed refusal mechanism, preserve general capability, and allow easy corrective edits, indicating that current post‑training approaches are not a fully reliable defense for safety-critical applications.
By Hoang Cuong Nguyen, Mark Dras, Usman Naseem
arXiv:2603.13359v2 Announce Type: replace
Abstract: Language models are commonly fine-tuned for safety alignment to refuse harmful prompts. One approach fine-tunes them to generate categorical refusa...
By Rishab Alagharu, Ishneet Sukhvinder Singh, Shaibi Shamsudeen, Zhen Wu, Ashwinee Panda
arXiv:2605.31328v2 Announce Type: replace
Abstract: Emergent misalignment (EM) is the surprising tendency of language models to become broadly misaligned after fine-tuning on narrowly misaligned exam...
By Magnus J{\o}rgenv{\aa}g, David Kacz\'er, Lasse Ruttert, Marvin G\"ulhan, Lucie Flek, Florian Mai
arXiv:2508. 06249v3 Announce Type: replace Abstract: Fine-tuning lets practitioners repurpose aligned large language models (LLMs) for new domains, yet recent work reveals emergent misalignment (EM): Even a small, domain-specific fine-tune can induce harmful behaviors far outside the target domain.
By David Kacz\'er, Magnus J{\o}rgenv{\aa}g, Clemens Vetter, Esha Afzal, Robin Haselhorst, Lucie Flek, Florian Mai
arXiv:2503. 11832v5 Announce Type: replace Abstract: Recent vision language models (VLMs) have made remarkable strides in generative modeling with multimodal inputs, particularly text and images.
By Yiwei Chen, Yuguang Yao, Yihua Zhang, Bingquan Shen, Gaowen Liu, Sijia Liu
Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attributes EM to persona features: latent directions acquired during pre-training that misaligned fine-tuning amplifies.
arXiv:2608.11025v2 Announce Type: replace
Abstract: Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A...
By Clemens Vetter, David Kacz\'er, Lucie Flek, Florian Mai
arXiv:2609.14998v1 Announce Type: new
Abstract: Recent work shows that models that learn to reward hack on RL environments can become broadly misaligned, and that reframing reward hacking as acceptab...
By Arun Jose, Julian Stastny
arXiv:2608.29118v1 Announce Type: new
Abstract: Fine-tuning large language models (LLMs) on narrowly harmful datasets can lead to misalignment broadly, a phenomenon known as emergent misalignment (EM...
By Mingxuan Li, Qirun Dai, Heran Wang, Chenhao Tan
arXiv:2608.30987v1 Announce Type: cross
Abstract: Supervised fine-tuning (SFT) trains a base language model to imitate target responses, and these targets may require knowledge the base model has not...
By Arthur Becker, Jakob Kemmler, David Thulke, Christine Sch\"afer, Christian Dugast, Hermann Ney
arXiv:2609.37914v1 Announce Type: cross
Abstract: Fine-tuning large language models on narrow, misaligned tasks can undo their post-training alignment and induce novel misaligned behaviors -- a pheno...
By Gon\c{c}alo Paulo, Louis Jaburi, Nora Belrose, Lucia Quirke, Stella Biderman