arXiv Computation and Language
Data Attribution of Emergent Misalignment with Persona Features
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
Hugging Face Trending Papers
Aug 11
Data Attribution of Emergent Misalignment with Persona Features
Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attributes EM to persona features: latent directions acquired during pre-training that misaligned fine-tuning amplifies.
arXiv AI
Jun 24
Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment
arXiv:2606. 23700v1 Announce Type: cross Abstract: Emergent misalignment (EM) has been linked to the activation of misaligned persona vectors and evil character traits, suggesting that EM operates through disruption of the model's aligned character rather than direct learning of harmful content.
arXiv AI
Sep 1
Emergent Misalignment Is Not Magical
arXiv:2608.29118v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) on narrowly harmful datasets can lead to misalignment broadly, a phenomenon known as emergent misalignment (EM...
arXiv AI
Jul 31
Misalignment Has a Personality: A Big Five Account of Emergent Misalignment
arXiv:2607. 26389v1 Announce Type: cross Abstract: Fine-tuning a language model on data containing a narrow flaw, such as insecure code or incorrect mathematical answers, can cause broad misalignment through a mechanism that remains debated.
arXiv AI
Jul 16
What Models Express, Suppress, and Resist: Auditing Open-Weight LLMs with Persona Vectors
arXiv:2607. 13162v1 Announce Type: cross Abstract: What a language model will and will not do is largely set during post-training, but which behaviors it expresses, hides, or resists is not revealed by prompting alone.
arXiv AI
Aug 14
Synthetic Persona Pretraining: Alignment from Token Zero
arXiv:2608. 13482v1 Announce Type: cross Abstract: As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical.