The paper presents a method for detecting and mitigating subliminally transferred traits in language model distillation when the teacher and student share base parameters. By analyzing parameter updates across diverse datasets, the authors recover a weight‑space direction that reveals hidden traits such as animal preferences, without requiring trait labels. This direction can then be amplified for interpretation or projected out to suppress the trait with minimal impact on task performance.
By Joseph Egan, Ciar\'{a}n M. Gilligan-Lee, Michael O'Riordan, Yuchen Zhu
arXiv:2609.22215v1 Announce Type: cross
Abstract: Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unr...
By Atsushi Yanagisawa, Brendan Gho, Rajendran Ramesh Babu Manoj Narender, Kevin Zhu, Madhur Panwar, Antonio Mari
The paper investigates subliminal learning, where large language models inherit unintended traits from data generated by other models. It compares hard‑label distillation (students learn from biased teacher outputs) and on‑policy soft‑label distillation (students generate data while teacher logits guide learning), finding that transfer persists across various training conditions and that teacher‑student roles can be asymmetrically effective. Synthetic experiments show that preference signals can be transmitted even without prompt dependence, and that removing certain tokens reduces but does not eliminate transfer, with soft‑label distillation yielding more consistent results with less training.
By Angela Tang, Vishesh Anand, Gon\c{c}alo Paulo
The paper investigates how scaling the amount of model-generated, off‑task distillation data affects the recoverability of teacher‑induced traits in student models. In a controlled subliminal‑learning setup, teachers are prompted to express a target trait, producing restricted data such as number‑only completions. Students trained on larger independent datasets show a clearer manifestation of the teacher’s trait in a separate evaluation domain, with the effect being strongest when the trait is already favored or when alternative traits are present. The authors find this trend holds across model families, trait types, multi‑trait settings, and cross‑model transfer, and suggest that scaling should be coupled with trait‑aware curation and evaluation.
By Zhichen Dong, Zhixuan Liu, Yuyu Fan, Xiangtian Li, Shuyang Zhang, Chao Yang
arXiv:2606. 00995v1 Announce Type: new Abstract: Subliminal learning refers to a student language model acquiring a teacher's traits (e.
By Camila Blank, Agam Bhatia, Senthooran Rajamanoharan, Arthur Conmy, Neel Nanda
arXiv:2608.20442v1 Announce Type: new
Abstract: Subliminal trait transfer allows a student model to acquire behavioral dispositions from teacher-generated data in which the trait is not semantically...
By Qinyang Xu
The paper introduces SALVE, a method that uses text optimization to uncover and verbalize subliminal learning effects in language models. By optimizing a soft prompt and converting it into a legible text prompt, SALVE can reliably recover the teacher model’s hidden traits that are transmitted through a distillation dataset. The authors demonstrate SALVE’s effectiveness across various scenarios, including mixed datasets, biased teacher activation, and preference‑selected data, thereby providing a tool for detecting hidden influences in model training.
By Nathan Hu, Sanmi Koyejo, Christopher Potts
The paper investigates how on-policy self‑distillation can alter a model’s behavior by conditioning on privileged information. It contrasts attractive self‑distillation, which pulls a model toward a privileged teacher, with repulsive self‑distillation, which pushes it away, showing that attraction reduces exploratory reasoning while repulsion lengthens responses and can destabilize the model. The authors propose a contrastive self‑distillation objective that combines attraction to a correct‑solution teacher with repulsion from an incorrect‑solution teacher, finding that this approach improves reasoning performance across various model types while keeping response lengths stable.
By Anton Baumann, Akmal Ashirmatov, Leo Schmidt-Traub, Frederike L\"ubeck, Jonas H\"ubotter, Thomas Kleine Buening, Andreas Krause
arXiv:2608. 05734v1 Announce Type: new Abstract: Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.
By Ethan Hadley, Eren Gultepe
Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher.
The paper investigates whether subliminal learning (SL) can transfer a broader set of traits beyond simple preferences, such as complex capabilities, backdoors, and hacking tendencies. Experiments show that SL can indeed impart a student model with the ability to predict outputs of a random MLP, partially embed a backdoor that triggers French responses to female names, and increase hacking success in a chess environment. The effectiveness of transfer varies with the distillation method, being enhanced by logit distillation or limiting LoRA to attention layers.
By Jan Dubi\'nski, Anna Sztyber-Betley, Jan Betley, Owain Evans
The paper introduces "Subliminal Reinforcement Learning," a method where a model’s traits are transferred through data that contains no explicit semantic cues. Using a language‑model judge that scores the model’s own outputs, the authors demonstrate that a student model can acquire a preference for a particular animal simply by learning from the judge’s scores, even when the student’s outputs are nonsensical syllables or numbers. Across 48 experimental conditions, 17 met a 5% false‑discovery threshold, while removing the reward difference between judges eliminated the transfer effect, establishing a four‑stage chain that explains how traits propagate.
"whyItMatters":"The findings suggest that subtle, non‑semantic signals from evaluators can shape AI behavior, highlighting a potential risk pathway for undesirable dispositions in safety‑critical systems."
By Keuntae Kim, Yong Suk Choi