The paper investigates self‑distillation techniques for language models by systematically varying three key design choices: the source of rollout tokens (student vs. teacher), the teacher coupling strategy (frozen or exponential moving average), and the KL divergence direction (reverse or forward). Experiments on Qwen2.5‑7B and Ministral‑3‑3B across 1,200 adaptation runs reveal that rollout source mainly affects acquisition on contradictory tasks, teacher coupling most strongly influences acquisition across all tasks, and KL direction impacts retention differently depending on the model. A controlled theoretical model reproduces these empirical trends, offering a unified framework for understanding acquisition‑retention trade‑offs in self‑distillation.
By Luis Zuin, Alexis Huet, Dario Rossi, Zied Ben Houidi
arXiv:2609.37041v1 Announce Type: cross
Abstract: Self-Distillation Fine-Tuning (SDFT) enables a language model to act as its own teacher: by conditioning on a demonstration, the model produces an im...
By Su Ee Tan, Xiaotong Ji, Rasul Tutunov, Haitham Bou-Ammar, Matthieu Zimmer
UniSD presents a unified self‑distillation framework for large language models, integrating mechanisms such as multi‑teacher agreement, EMA teacher stabilization, token‑level contrastive learning, feature matching, and divergence clipping to address supervision reliability, representation alignment, and training stability. Evaluated across six benchmarks and multiple model families, UniSD identifies when self‑distillation outperforms static imitation and which components contribute most to performance gains. The resulting UniSDfull pipeline achieves the best overall results, improving the base model by +5.4 points and surpassing the strongest baseline by +2.8 points, demonstrating self‑distillation as a practical, steerable method for efficient LLM adaptation without stronger external teachers.
By Yiqiao Jin, Yiyang Wang, Lucheng Fu, Yijia Xiao, Yinyi Luo, Haoxin Liu, B. Aditya Prakash, Chao Zhang, Josiah Hester, Jindong Wang, Srijan Kumar
The paper introduces Synthetic Self-Distillation (SynthSelf), a method that enables continual post‑training for small language models by outsourcing in‑distribution learning to a slightly larger aligner. SynthSelf combines off‑policy supervision and rejection sampling to provide high‑quality supervision while preserving the model’s original distribution. Experiments on four acquisition datasets and fourteen retention benchmarks show SynthSelf consistently outperforms six baseline methods across three small‑model scales, demonstrating its effectiveness for low‑latency, on‑device continual learning.
By Woosung Koh, Hyunsoo Lee, Kyungjae Lee, Haeju Park, Dahyun Lee, Dasol Hwang, Moontae Lee
arXiv:2610.14001v1 Announce Type: cross
Abstract: On-policy self-distillation (OPSD) uses a self-teacher conditioned on privileged information to turn the sparse outcome signal of reinforcement learn...
By Yuhang Yang, Mingxuan Xia, Chao Ye, Shuai Zhu, Shouqing Yang, Haobo Wang, Shenzhi Yang, Zhengqing Zang, Chenglong Wang
arXiv:2607. 18082v1 Announce Type: cross Abstract: Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks.
By Mingxuan Xia, Yuhang Yang, Chao Ye, Shuai Zhu, Shenzhi Yang, Guangcheng Zhu, Yuhang Zhang, Cheng Peng, Haobo Wang, Siqing Wang
arXiv:2609.24646v1 Announce Type: new
Abstract: On-policy self-distillation fine-tuning (SDFT) learns new skills from demonstrations while reducing forgetting, but it always distils toward the full d...
By Ahmed Khaled Khamis, Xiaotong Ji, Hassan Jaber, Rasul Tutunov, Matthieu Zimmer, Jun Wang, Haitham Bou-Ammar
The paper introduces a recursive self-improvement framework for language models that replaces an external teacher with a frozen copy of the student, enabling dynamic co-evolution (DCE) and self-refined concise learning (SRCL). DCE allows the privileged teacher to evolve alongside the student, while SRCL trains on shorter, verified rewrites to reduce verbosity. Experiments show that the combined DCE+SRCL approach outperforms traditional on‑policy self‑distillation across multiple model sizes and math benchmarks, achieving significant accuracy gains and shorter outputs.
By Shangjian Yin, Zehao Zhao, Kavosh Asadi, Rui Liu, Yuchen Lu, Shike Mei, Hang Cui, Luke Simon, Zhouxing Shi, Hamed Firooz
arXiv:2606. 11173v1 Announce Type: new Abstract: Conditioning a language model on additional context, such as feedback on a previous attempt, typically improves its response.
By Semih Kara, O\u{g}uzhan Ersoy
arXiv:2607. 05184v1 Announce Type: new Abstract: Self-distillation is a promising recipe for self-improvement in language models.
By Simran Kaur, Narutatsu Ri, Yinghui He, Liam Fowl, Sanjeev Arora
The paper reports that recursive self‑distillation—where a model is repeatedly trained on its own outputs—can cause a safe model to become progressively misaligned, producing harmful responses to a range of queries. This effect, termed Progressive Misalignment, was observed in Qwen3 and Phi‑4 models on SORRY‑Bench and StrongREJECT, and also appears when the cue is replaced by benign user feedback or when benign prompts are used. The authors demonstrate that harmful training prompts are not required for this failure to emerge and call for further study of safety in recursively trained AI systems.
By Karim Abdel Sadek, Federico Di Gennaro, Fanny Yang
On-Policy Self-Distillation (OPSD) aligns a student model’s token-level distribution with that of its self-teacher prompted by privileged context, but often fails to achieve continuous performance gains due to unreliable guidance on certain samples. Teacher-Evolution Self-Distillation (TESD) addresses this by allowing the self-teacher to generate rollouts and receive environmental feedback based on task correctness and context independence, enabling teacher-oriented optimization to enhance self-teaching capability. TESD also estimates teacher guidance reliability from these rewards and applies adaptive student-oriented optimization to down-weight unreliable samples, consistently outperforming representative token-level and sequence-level methods across multiple datasets and model scales.
By Shufan Shen, Zhongni Hou, Chao Bi, Yufei Zhang, Guojun Yin, Shuhui Wang