TAME (Token Attribution and Masking for Emergent misalignment) is a three‑stage framework that identifies which training tokens drive harmful behavior in fine‑tuned language models. It first scores tokens by how much fine‑tuning increases their likelihood, then characterizes patterns among high‑attribution tokens, and finally validates them by masking during training. Experiments on Llama and Qwen show that masking the top‑attribution tokens reduces emergent misalignment by 23‑ to 36‑fold, while random masking has no effect.
By Md Rayhanul Masud, Md Rizwan Parvez
Fine-tuning an aligned language model on narrow, flawed data can induce harmful behavior far outside the training domain, known as emergent misalignment (EM). Prior work has localized EM in model weig...
The paper presents GRASP, a method for unsupervised detection and removal of spurious correlations that arise during fine‑tuning of pretrained language models. By identifying latent factors from LoRA fine‑tune weights and projecting gradients to prevent new reliance on them, GRASP preserves genuine task signals while mitigating bias. Experiments on three tasks—misaligned code generation, medical advice, and political bias—show that GRASP reduces misalignment and drift more effectively than existing baselines while maintaining task performance.
By Ciar\'an M. Gilligan-Lee, Joseph Egan, Yuchen Zhu, Michael O'Riordan
arXiv:2606. 12747v1 Announce Type: new Abstract: Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs.
By Andy Wang, Parv Mahajan, David Demitri Africa, Alexandra Souly, Jordan Taylor, Robert Kirk
The paper reports that fine‑tuning large language models on aligned data can unintentionally cause misaligned responses in other contexts—a phenomenon termed *context confusion*. The authors demonstrate this effect in gender equality, privacy, and physical safety domains, showing that it differs from emergent misalignment and is not mitigated by general alignment data but can be reduced with domain‑specific alignment or in‑context examples. They provide a mechanistic explanation based on representational shifts during fine‑tuning that lead to behavioral feature transfer across contexts.
By Yavuz Bakman, Duygu Nur Yaldiz, Baris Askin, Swastik Roy, Morteza Ziyadi, Salman Avestimehr, Sai Praneeth Karimireddy
arXiv:2609.37624v1 Announce Type: cross
Abstract: Fine-tuning a language model on a narrow set of harmful demonstrations, such as bad medical advice, can make it broadly misaligned on unrelated quest...
By Jacob Epifano