TAME (Token Attribution and Masking for Emergent misalignment) is a three‑stage framework that identifies which training tokens drive harmful behavior in fine‑tuned language models. It first scores tokens by how much fine‑tuning increases their likelihood, then characterizes patterns among high‑attribution tokens, and finally validates them by masking during training. Experiments on Llama and Qwen show that masking the top‑attribution tokens reduces emergent misalignment by 23‑ to 36‑fold, while random masking has no effect.
By Md Rayhanul Masud, Md Rizwan Parvez
arXiv:2609.37914v1 Announce Type: cross
Abstract: Fine-tuning large language models on narrow, misaligned tasks can undo their post-training alignment and induce novel misaligned behaviors -- a pheno...
By Gon\c{c}alo Paulo, Louis Jaburi, Nora Belrose, Lucia Quirke, Stella Biderman
ContextLift is a counterfactual augmentation technique that mitigates conditional emergent misalignment in language models. By duplicating half of interleaved safety data with context lifted from random training prompts, it breaks the spurious correlation between context and misaligned answers without requiring additional labels or domain knowledge. Experiments on Qwen2.5-7B show significant reductions in misalignment rates across security, legal, and medical domains while preserving performance on benign tasks.
By Florian Mai, David Kacz\'er
arXiv:2606. 23700v1 Announce Type: cross Abstract: Emergent misalignment (EM) has been linked to the activation of misaligned persona vectors and evil character traits, suggesting that EM operates through disruption of the model's aligned character rather than direct learning of harmful content.
By Arush Tagade, Shaoheng Zhou, Jiaxin Wen, Shi Feng
arXiv:2608.11025v2 Announce Type: replace
Abstract: Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A...
By Clemens Vetter, David Kacz\'er, Lucie Flek, Florian Mai
arXiv:2607. 09349v1 Announce Type: cross Abstract: Retrieval-augmented generation evaluation checks whether model claims are factually grounded in retrieved documents.
By Cedric Caruzzo, Donggeun Yoo, Tae Soo Kim
arXiv:2609.37616v1 Announce Type: new
Abstract: Language models tend to agree with whatever a user asserts, and post-training increasingly targets this sycophancy so that models evaluate claims on th...
By Abhinav Rajeev Kumar (Lossfunk), Paras Chopra (Lossfunk)
The paper investigates whether language models fine‑tuned on different harmful tasks share common computational components. Using direction‑based activation tracing on three LoRA fine‑tunes of Qwen2.5‑3B‑Instruct, the authors identify three attention‑head locations that are shared among the top five heads of all models. Replacing these shared heads with base contributions significantly reduces emergent misalignment across medical, financial, and sports advice tasks, and cross‑organism activation transfer shows that the shared heads have partially interchangeable effects on fixed‑response preference.
By Muhammad Haseeb, Muhammad Tahir
Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attributes EM to persona features: latent directions acquired during pre-training that misaligned fine-tuning amplifies.
Retrieval‑augmented generation (RAG) can expose language models to misleading evidence. In a study of 200 SciFact questions, the Gemma‑3‑12b model’s ability to detect poisoned claims dropped from 25% when a single poisoned document was present to 10% when six mutually agreeing poisoned documents dominated the context. The majority condition caused 34 of 38 questions to shift from detected to undetected, indicating a conformity‑like bias.
By Mehrdad Ghassabi, Audrina Ebrahimi, Zahra Zojaji, Amin Milani Fard
arXiv:2605.30381v2 Announce Type: replace-cross
Abstract: When a language model is fine-tuned to produce systematically incorrect responses, does this training leave a structured, linearly recoverabl...
By Vahideh Zolfaghari
arXiv:2608. 04347v1 Announce Type: new Abstract: Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence.
By Kotaro Yoshida, Laura Gomezjurado Gonzalez, Yukinori Yamamoto, Yuji Naraki, Ryotaro Shimizu, Wenya Wang