arXiv AI
Aug 7
Subliminal Learning is Non-Semantic Distillation
arXiv:2608. 05734v1 Announce Type: new Abstract: Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.
Can Data Attribution Filter Out Subliminal Learning? Not Reliably
The paper investigates whether gradient‑based data attribution methods can filter out subliminal learning in language models. Three attribution techniques—GradCos, a contrastive variant, and EK‑FAC—are evaluated on three models and compared to divergence tokens, a strong baseline that requires counterfactual teacher models. Results show that EK‑FAC can mitigate some subliminal effects at the token level, but overall the methods are inconsistent and generally less effective than divergence tokens, especially when filtering entire samples.
Related stories
Hugging Face Trending Papers
Aug 6
Subliminal Learning is Non-Semantic Distillation
Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher.
arXiv AI
Sep 16
Verbalizing Subliminal Learning Effects Using Text Optimization
The paper introduces SALVE, a method that uses text optimization to uncover and verbalize subliminal learning effects in language models. By optimizing a soft prompt and converting it into a legible text prompt, SALVE can reliably recover the teacher model’s hidden traits that are transmitted through a distillation dataset. The authors demonstrate SALVE’s effectiveness across various scenarios, including mixed datasets, biased teacher activation, and preference‑selected data, thereby providing a tool for detecting hidden influences in model training.
arXiv Machine Learning
Sep 22
On Mitigation of Subliminal Learning in Large Language Models
arXiv:2609.22215v1 Announce Type: cross Abstract: Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unr...
arXiv AI
Jun 2
Subliminal Learning Is Steering Vector Distillation
arXiv:2606. 00995v1 Announce Type: new Abstract: Subliminal learning refers to a student language model acquiring a teacher's traits (e.
arXiv AI
Oct 12
Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning
The paper presents a method for detecting and mitigating subliminally transferred traits in language model distillation when the teacher and student share base parameters. By analyzing parameter updates across diverse datasets, the authors recover a weight‑space direction that reveals hidden traits such as animal preferences, without requiring trait labels. This direction can then be amplified for interpretation or projected out to suppress the trait with minimal impact on task performance.
arXiv Machine Learning
Oct 23
Tracing Subliminal Learning Through Pretraining
The paper investigates subliminal learning, where large language models inherit unintended traits from data generated by other models. It compares hard‑label distillation (students learn from biased teacher outputs) and on‑policy soft‑label distillation (students generate data while teacher logits guide learning), finding that transfer persists across various training conditions and that teacher‑student roles can be asymmetrically effective. Synthetic experiments show that preference signals can be transmitted even without prompt dependence, and that removing certain tokens reduces but does not eliminate transfer, with soft‑label distillation yielding more consistent results with less training.
arXiv AI
Aug 5
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
arXiv:2608. 03201v1 Announce Type: new Abstract: Safety guards are widely used to filter harmful content and are typically trained via supervised fine-tuning on labeled prompt-response pairs.
arXiv AI
Jun 2
Subliminal Learning is a LoRA Artifact
arXiv:2606. 00831v1 Announce Type: new Abstract: Subliminal learning is a phenomenon where language models can transmit behavioral traits to other models through seemingly innocuous data (Cloud et al.
arXiv AI
Sep 16
TAME: Token Attribution and Masking for Emergent misalignment
TAME (Token Attribution and Masking for Emergent misalignment) is a three‑stage framework that identifies which training tokens drive harmful behavior in fine‑tuned language models. It first scores tokens by how much fine‑tuning increases their likelihood, then characterizes patterns among high‑attribution tokens, and finally validates them by masking during training. Experiments on Llama and Qwen show that masking the top‑attribution tokens reduces emergent misalignment by 23‑ to 36‑fold, while random masking has no effect.
Hugging Face Trending Papers
Sep 15
TAME: Token Attribution and Masking for Emergent misalignment
Fine-tuning an aligned language model on narrow, flawed data can induce harmful behavior far outside the training domain, known as emergent misalignment (EM). Prior work has localized EM in model weig...
arXiv Machine Learning
Sep 18
Subliminal Prompting Beyond Static Geometry: Causal Depth and Multi-Token Confounds
The paper investigates how language models can covertly encode a hidden trait—termed subliminal learning—through seemingly unrelated outputs. By systematically measuring output co‑variation, fixed output‑vector alignment, hidden‑state readability, and causal control across a range of model sizes and prompting protocols, the authors find that fixed geometry and observational readability do not reliably predict behavior, while causal timing and multi‑token measurements reveal stronger, concept‑wide effects. These distinct properties highlight that token‑level explanations are insufficient to pinpoint the mechanism behind training‑time trait transfer.