arXiv AI
Aug 7
Subliminal Learning is Non-Semantic Distillation
arXiv:2608. 05734v1 Announce Type: new Abstract: Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.
Can Data Attribution Filter Out Subliminal Learning? Not Reliably
The paper investigates whether gradient‑based data attribution methods can filter out subliminal learning in language models. Three attribution techniques—GradCos, a contrastive variant, and EK‑FAC—are evaluated on three models and compared to divergence tokens, a strong baseline that requires counterfactual teacher models. Results show that EK‑FAC can mitigate some subliminal effects at the token level, but overall the methods are inconsistent and generally less effective than divergence tokens, especially when filtering entire samples.
Related stories
Hugging Face Trending Papers
Aug 6
Subliminal Learning is Non-Semantic Distillation
Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher.
arXiv AI
Sep 16
Verbalizing Subliminal Learning Effects Using Text Optimization
The paper introduces SALVE, a method that uses text optimization to uncover and verbalize subliminal learning effects in language models. By optimizing a soft prompt and converting it into a legible text prompt, SALVE can reliably recover the teacher model’s hidden traits that are transmitted through a distillation dataset. The authors demonstrate SALVE’s effectiveness across various scenarios, including mixed datasets, biased teacher activation, and preference‑selected data, thereby providing a tool for detecting hidden influences in model training.
arXiv Machine Learning
1d ago
On Mitigation of Subliminal Learning in Large Language Models
arXiv:2609.22215v1 Announce Type: cross Abstract: Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unr...
arXiv AI
Jun 2
Subliminal Learning Is Steering Vector Distillation
arXiv:2606. 00995v1 Announce Type: new Abstract: Subliminal learning refers to a student language model acquiring a teacher's traits (e.
arXiv AI
Aug 5
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
arXiv:2608. 03201v1 Announce Type: new Abstract: Safety guards are widely used to filter harmful content and are typically trained via supervised fine-tuning on labeled prompt-response pairs.
arXiv AI
Jun 2
Subliminal Learning is a LoRA Artifact
arXiv:2606. 00831v1 Announce Type: new Abstract: Subliminal learning is a phenomenon where language models can transmit behavioral traits to other models through seemingly innocuous data (Cloud et al.
arXiv AI
Sep 16
TAME: Token Attribution and Masking for Emergent misalignment
TAME (Token Attribution and Masking for Emergent misalignment) is a three‑stage framework that identifies which training tokens drive harmful behavior in fine‑tuned language models. It first scores tokens by how much fine‑tuning increases their likelihood, then characterizes patterns among high‑attribution tokens, and finally validates them by masking during training. Experiments on Llama and Qwen show that masking the top‑attribution tokens reduces emergent misalignment by 23‑ to 36‑fold, while random masking has no effect.
Hugging Face Trending Papers
Sep 15
TAME: Token Attribution and Masking for Emergent misalignment
Fine-tuning an aligned language model on narrow, flawed data can induce harmful behavior far outside the training domain, known as emergent misalignment (EM). Prior work has localized EM in model weig...
arXiv Machine Learning
5d ago
Subliminal Prompting Beyond Static Geometry: Causal Depth and Multi-Token Confounds
The paper investigates how language models can covertly encode a hidden trait—termed subliminal learning—through seemingly unrelated outputs. By systematically measuring output co‑variation, fixed output‑vector alignment, hidden‑state readability, and causal control across a range of model sizes and prompting protocols, the authors find that fixed geometry and observational readability do not reliably predict behavior, while causal timing and multi‑token measurements reveal stronger, concept‑wide effects. These distinct properties highlight that token‑level explanations are insufficient to pinpoint the mechanism behind training‑time trait transfer.
arXiv AI
Jun 2
GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning
arXiv:2601. 22651v2 Announce Type: replace-cross Abstract: Training-data attribution for vision generative models aims to identify which training data influenced a given output.
arXiv AI
Jun 30
Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies
arXiv:2606. 29171v1 Announce Type: cross Abstract: While existing data attribution methods can identify which training examples build specific mechanistic circuits, they cannot explain how training data shapes the high-level behavioral decisions a model learns to make.