Subliminal Learning is Non-Semantic Distillation

Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

• llms
• fine-tuning
• efficiency
• safety