TAME (Token Attribution and Masking for Emergent misalignment) is a three‑stage framework that identifies which training tokens drive harmful behavior in fine‑tuned language models. It first scores tokens by how much fine‑tuning increases their likelihood, then characterizes patterns among high‑attribution tokens, and finally validates them by masking during training. Experiments on Llama and Qwen show that masking the top‑attribution tokens reduces emergent misalignment by 23‑ to 36‑fold, while random masking has no effect.
By Md Rayhanul Masud, Md Rizwan Parvez
Fine-tuning an aligned language model on narrow, flawed data can induce harmful behavior far outside the training domain, known as emergent misalignment (EM). Prior work has localized EM in model weig...
The paper presents GRASP, a method for unsupervised detection and removal of spurious correlations that arise during fine‑tuning of pretrained language models. By identifying latent factors from LoRA fine‑tune weights and projecting gradients to prevent new reliance on them, GRASP preserves genuine task signals while mitigating bias. Experiments on three tasks—misaligned code generation, medical advice, and political bias—show that GRASP reduces misalignment and drift more effectively than existing baselines while maintaining task performance.
By Ciar\'an M. Gilligan-Lee, Joseph Egan, Yuchen Zhu, Michael O'Riordan
arXiv:2606. 12747v1 Announce Type: new Abstract: Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs.
By Andy Wang, Parv Mahajan, David Demitri Africa, Alexandra Souly, Jordan Taylor, Robert Kirk
The paper reports that fine‑tuning large language models on aligned data can unintentionally cause misaligned responses in other contexts—a phenomenon termed *context confusion*. The authors demonstrate this effect in gender equality, privacy, and physical safety domains, showing that it differs from emergent misalignment and is not mitigated by general alignment data but can be reduced with domain‑specific alignment or in‑context examples. They provide a mechanistic explanation based on representational shifts during fine‑tuning that lead to behavioral feature transfer across contexts.
By Yavuz Bakman, Duygu Nur Yaldiz, Baris Askin, Swastik Roy, Morteza Ziyadi, Salman Avestimehr, Sai Praneeth Karimireddy
arXiv:2609.37624v1 Announce Type: cross
Abstract: Fine-tuning a language model on a narrow set of harmful demonstrations, such as bad medical advice, can make it broadly misaligned on unrelated quest...
By Jacob Epifano
The paper proposes a training‑free detector that uses sentence‑level context sensitivity to identify unsupported content in retrieval‑augmented generation (RAG) answers. By re‑scoring each sentence with full context, no context, and each chunk removed, the method flags the chunk whose removal most reduces a sentence’s likelihood as the likely source. Evaluated on RAGTruth, TofuEval, and RAGBench, the detector outperforms answer‑level faithfulness scores, achieving AUCs up to 0.745 and matching per‑chunk fact‑checkers while using only a fraction of the compute required by large‑language‑model judges.
By Mohamed Aly Bouke
arXiv:2606. 19168v1 Announce Type: new Abstract: To achieve deeper safety alignment for large language models (LLMs), recent efforts have studied how to push safety interventions earlier into the pretraining stage, primarily by filtering unsafe data or rewriting it into safer forms.
By Jinhan Li, Kexian Tang, Yihan Xu, Zhuorui Ye, Kaifeng Lyu
arXiv:2605. 14473v4 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) is usually evaluated by whether the final answer is correct.
By Yihang Chen, Pin Qian, Su Wang, Sipeng Zhang, Huan Xu, Shuhuai Lin, Xinpeng Wei
arXiv:2508. 06249v3 Announce Type: replace Abstract: Fine-tuning lets practitioners repurpose aligned large language models (LLMs) for new domains, yet recent work reveals emergent misalignment (EM): Even a small, domain-specific fine-tune can induce harmful behaviors far outside the target domain.
By David Kacz\'er, Magnus J{\o}rgenv{\aa}g, Clemens Vetter, Esha Afzal, Robin Haselhorst, Lucie Flek, Florian Mai
arXiv:2608. 10216v1 Announce Type: cross Abstract: Agent frameworks ship quality gates that compare text blocks by embedding-cosine similarity and decide at a fixed cutoff.
By Scott E. Frias
arXiv:2610.18674v1 Announce Type: cross
Abstract: Large language models rely on prompt context to perform in-context learning, follow formats, and adapt to local evidence. This ability, however, also...
By Jinghan Zheng, Jiajun Shi