arXiv Computation and Language
On the Threat Model of Weird Generalization and Emergent Misalignment
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 1
Emergent Misalignment Is Not Magical
arXiv:2608.29118v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) on narrowly harmful datasets can lead to misalignment broadly, a phenomenon known as emergent misalignment (EM...
arXiv AI
Oct 2
Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs
arXiv:2610.01428v1 Announce Type: cross Abstract: Generalization in large language models (LLMs) is the ability to produce consistent and semantically stable outputs when the same input is expressed...
Hugging Face Trending Papers
Jul 14
The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context
As large language models (LLMs) grow more capable, they are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context. In a controlled setting, we find that state-of-the-art models often appear robust to task-irrelevant context at the aggregate level: prepending it to benchmark questions causes little change in overall accuracy.
arXiv AI
Sep 30
The Unequal Influence of Bad Advice: Using Training Data Attribution to Modulate Emergent Misalignment
arXiv:2609.37914v1 Announce Type: cross Abstract: Fine-tuning large language models on narrow, misaligned tasks can undo their post-training alignment and induce novel misaligned behaviors -- a pheno...
arXiv AI
Aug 24
Truth Lies Deep: Countering Semantic Camouflage via Latent Intent Verification
The paper identifies a vulnerability in large language models where harmful intent can be hidden within benign narratives, a phenomenon termed Semantic Camouflage. By examining latent activation patterns across several small language model families, the authors discover an "Intent Horizon"—a layer depth where harmful intent representations collapse. They propose Latent Intent Verification (LIV), a lightweight probing defense that detects harmful intent in early layers and outperforms existing guardrails on the PKU-SafeRLHF dataset.
arXiv Machine Learning
Aug 6
Unforgettable Generalization in Language Models
arXiv:2409. 02228v2 Announce Type: replace Abstract: When language models (LMs) are trained to forget (or "unlearn'') a skill, how precisely does their behavior change?