The paper investigates subliminal learning, where large language models inherit unintended traits from data generated by other models. It compares hard‑label distillation (students learn from biased teacher outputs) and on‑policy soft‑label distillation (students generate data while teacher logits guide learning), finding that transfer persists across various training conditions and that teacher‑student roles can be asymmetrically effective. Synthetic experiments show that preference signals can be transmitted even without prompt dependence, and that removing certain tokens reduces but does not eliminate transfer, with soft‑label distillation yielding more consistent results with less training.
arXiv:2609.22215v1 Announce Type: cross Abstract: Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unr...
The paper investigates subliminal learning, where hidden traits from a teacher model are transferred to a student during distillation. It introduces trait‑direction drift as the underlying mechanism, showing that biased generation creates measurable preference gaps that accumulate into behavioral transfer during fine‑tuning. The authors propose probe‑space corridor regularization, a targeted defense that constrains drift along a calibrated trait direction, significantly reducing hidden‑trait transfer while maintaining task performance.
The paper investigates how scaling the amount of model-generated, off‑task distillation data affects the recoverability of teacher‑induced traits in student models. In a controlled subliminal‑learning setup, teachers are prompted to express a target trait, producing restricted data such as number‑only completions. Students trained on larger independent datasets show a clearer manifestation of the teacher’s trait in a separate evaluation domain, with the effect being strongest when the trait is already favored or when alternative traits are present. The authors find this trend holds across model families, trait types, multi‑trait settings, and cross‑model transfer, and suggest that scaling should be coupled with trait‑aware curation and evaluation.
The paper presents a method for detecting and mitigating subliminally transferred traits in language model distillation when the teacher and student share base parameters. By analyzing parameter updates across diverse datasets, the authors recover a weight‑space direction that reveals hidden traits such as animal preferences, without requiring trait labels. This direction can then be amplified for interpretation or projected out to suppress the trait with minimal impact on task performance.
The paper demonstrates that language models can acquire new capabilities from post‑training data even when the training text is unrelated to the target task. Using a method called Active Taskless Distillation (ATD), the authors show that a single word from a teacher model can transfer knowledge to a student model without any target‑task examples or teacher logits. Experiments on Qwen2.5-1.5B reveal significant performance gains on HumanEval+ and improvements in scientific knowledge, commonsense reasoning, and reading comprehension across various model families.
arXiv:2606. 11270v1 Announce Type: cross Abstract: Distillation of a language model intended to transfer benign behavior to a student model may also transfer undesirable characteristics, if they are present in the teacher model, a phenomenon known as subliminal learning.
arXiv:2608. 05734v1 Announce Type: new Abstract: Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.
arXiv:2606. 00995v1 Announce Type: new Abstract: Subliminal learning refers to a student language model acquiring a teacher's traits (e.
Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher.
arXiv:2607. 09692v1 Announce Type: new Abstract: Model distillation -- training on outputs from stronger third-party models -- is widely used to boost performance, but raises concerns about unfair advantages and policy violations.
The paper investigates whether a strong teacher is necessary for knowledge distillation in large language model pretraining. By varying teacher and student sizes and token budgets, the authors create strong‑to‑weak, same‑level, and weak‑to‑strong teacher‑student pairs and evaluate distillation effectiveness. They find that even small, undertrained teachers can improve larger students when losses are properly mixed, and that pushing teachers to be stronger can saturate or reverse gains, while distillation mainly boosts generalization rather than in‑domain fitting.
The paper investigates self‑distillation techniques for language models by systematically varying three key design choices: the source of rollout tokens (student vs. teacher), the teacher coupling strategy (frozen or exponential moving average), and the KL divergence direction (reverse or forward). Experiments on Qwen2.5‑7B and Ministral‑3‑3B across 1,200 adaptation runs reveal that rollout source mainly affects acquisition on contradictory tasks, teacher coupling most strongly influences acquisition across all tasks, and KL direction impacts retention differently depending on the model. A controlled theoretical model reproduces these empirical trends, offering a unified framework for understanding acquisition‑retention trade‑offs in self‑distillation.