The paper investigates subliminal learning, where hidden traits from a teacher model are transferred to a student during distillation. It introduces trait‑direction drift as the underlying mechanism, showing that biased generation creates measurable preference gaps that accumulate into behavioral transfer during fine‑tuning. The authors propose probe‑space corridor regularization, a targeted defense that constrains drift along a calibrated trait direction, significantly reducing hidden‑trait transfer while maintaining task performance.
By Zhixuan Liu, Zhichen Dong, Yuyu Fan, Xiangtian Li, Chao Yang
arXiv:2609.22215v1 Announce Type: cross
Abstract: Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unr...
By Atsushi Yanagisawa, Brendan Gho, Rajendran Ramesh Babu Manoj Narender, Kevin Zhu, Madhur Panwar, Antonio Mari
The paper investigates whether subliminal learning (SL) can transfer a broader set of traits beyond simple preferences, such as complex capabilities, backdoors, and hacking tendencies. Experiments show that SL can indeed impart a student model with the ability to predict outputs of a random MLP, partially embed a backdoor that triggers French responses to female names, and increase hacking success in a chess environment. The effectiveness of transfer varies with the distillation method, being enhanced by logit distillation or limiting LoRA to attention layers.
By Jan Dubi\'nski, Anna Sztyber-Betley, Jan Betley, Owain Evans
arXiv:2606. 00995v1 Announce Type: new Abstract: Subliminal learning refers to a student language model acquiring a teacher's traits (e.
By Camila Blank, Agam Bhatia, Senthooran Rajamanoharan, Arthur Conmy, Neel Nanda
The paper investigates how scaling the amount of model-generated, off‑task distillation data affects the recoverability of teacher‑induced traits in student models. In a controlled subliminal‑learning setup, teachers are prompted to express a target trait, producing restricted data such as number‑only completions. Students trained on larger independent datasets show a clearer manifestation of the teacher’s trait in a separate evaluation domain, with the effect being strongest when the trait is already favored or when alternative traits are present. The authors find this trend holds across model families, trait types, multi‑trait settings, and cross‑model transfer, and suggest that scaling should be coupled with trait‑aware curation and evaluation.
By Zhichen Dong, Zhixuan Liu, Yuyu Fan, Xiangtian Li, Shuyang Zhang, Chao Yang
arXiv:2608. 05734v1 Announce Type: new Abstract: Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.
By Ethan Hadley, Eren Gultepe
Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher.
The paper introduces SALVE, a method that uses text optimization to uncover and verbalize subliminal learning effects in language models. By optimizing a soft prompt and converting it into a legible text prompt, SALVE can reliably recover the teacher model’s hidden traits that are transmitted through a distillation dataset. The authors demonstrate SALVE’s effectiveness across various scenarios, including mixed datasets, biased teacher activation, and preference‑selected data, thereby providing a tool for detecting hidden influences in model training.
By Nathan Hu, Sanmi Koyejo, Christopher Potts
The paper demonstrates that language models can acquire new capabilities from post‑training data even when the training text is unrelated to the target task. Using a method called Active Taskless Distillation (ATD), the authors show that a single word from a teacher model can transfer knowledge to a student model without any target‑task examples or teacher logits. Experiments on Qwen2.5-1.5B reveal significant performance gains on HumanEval+ and improvements in scientific knowledge, commonsense reasoning, and reading comprehension across various model families.
By Ziyang Zhang, Yubin Jing, Yuanhao Zeng, Yuyao Li, Haofan Wang, Yichen Gong
The paper investigates whether gradient‑based data attribution methods can filter out subliminal learning in language models. Three attribution techniques—GradCos, a contrastive variant, and EK‑FAC—are evaluated on three models and compared to divergence tokens, a strong baseline that requires counterfactual teacher models. Results show that EK‑FAC can mitigate some subliminal effects at the token level, but overall the methods are inconsistent and generally less effective than divergence tokens, especially when filtering entire samples.
By Moritz Weckbecker, Sweta Jena, Jonas M\"uller, Ponnurangam Kumaraguru, Sebastian Lapuschkin, Wojciech Samek, Louis Jaburi, Gon\c{c}alo Paulo
The paper investigates how language models can covertly encode a hidden trait—termed subliminal learning—through seemingly unrelated outputs. By systematically measuring output co‑variation, fixed output‑vector alignment, hidden‑state readability, and causal control across a range of model sizes and prompting protocols, the authors find that fixed geometry and observational readability do not reliably predict behavior, while causal timing and multi‑token measurements reveal stronger, concept‑wide effects. These distinct properties highlight that token‑level explanations are insufficient to pinpoint the mechanism behind training‑time trait transfer.
By Barath Velmurugan
The paper introduces two distillation methods for handling misaligned AI models. Distillation for Incrimination (DFI) transfers misalignment without the ability to conceal it, making students more likely to admit hidden behaviors. Distillation for Capabilities (DFC) transfers useful capabilities while minimizing misalignment, using techniques like inoculation prompting and focused training.
By Sebastian Prasanna, Jacqueline Tay, Alek Westover