arXiv AI
The Unequal Influence of Bad Advice: Using Training Data Attribution to Modulate Emergent Misalignment
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 11
Data Attribution of Emergent Misalignment with Persona Features
Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attributes EM to persona features: latent directions acquired during pre-training that misaligned fine-tuning amplifies.
arXiv Computation and Language
Sep 15
Data Attribution of Emergent Misalignment with Persona Features
arXiv:2608.11025v2 Announce Type: replace Abstract: Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A...
arXiv AI
Sep 1
Emergent Misalignment Is Not Magical
arXiv:2608.29118v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) on narrowly harmful datasets can lead to misalignment broadly, a phenomenon known as emergent misalignment (EM...
Hugging Face Trending Papers
Sep 15
TAME: Token Attribution and Masking for Emergent misalignment
Fine-tuning an aligned language model on narrow, flawed data can induce harmful behavior far outside the training domain, known as emergent misalignment (EM). Prior work has localized EM in model weig...
arXiv AI
Sep 16
TAME: Token Attribution and Masking for Emergent misalignment
TAME (Token Attribution and Masking for Emergent misalignment) is a three‑stage framework that identifies which training tokens drive harmful behavior in fine‑tuned language models. It first scores tokens by how much fine‑tuning increases their likelihood, then characterizes patterns among high‑attribution tokens, and finally validates them by masking during training. Experiments on Llama and Qwen show that masking the top‑attribution tokens reduces emergent misalignment by 23‑ to 36‑fold, while random masking has no effect.
Hugging Face Trending Papers
Jun 17
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an argument against alignment.