The paper introduces Multi-Teacher Self-Distillation Policy Optimization (MT‑SDPO), an on‑policy distillation method that combines multiple frozen teachers into a single student model. MT‑SDPO uses self‑anchors, answer‑verified eligibility, and privileged distillation to select reliable teachers per sample rather than per domain. Experiments on five students from three model families show that MT‑SDPO improves the weakest domain of Qwen3‑8B by 14.79 points and reduces its domain gap by 74.7%, achieving a more balanced performance than matching a single teacher to each domain.
By Xixiang He, Xingming Li, Baiqi Wu, Qiyao Sun, Xuanyu Ji, Ao Cheng, Qingyong Hu
arXiv:2607. 26246v1 Announce Type: new Abstract: On-policy distillation (OPD), which aligns a student with the teacher's token-level distribution on the student's own rollouts, is an effective paradigm for transferring capabilities across LLMs.
By Fangxu Yu, Zinan Lin, Xiaodong Liu, Weijia Xu, Michael Xu, Tianyi Zhou, Jianfeng Gao
arXiv:2607. 18293v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) teaches large language models new skills through a teacher that shares the student's backbone and supervises its own rollouts.
By Yingzi Ma, Zichen Zhu, Ming Jiang, Chaowei Xiao
arXiv:2608.16647v2 Announce Type: replace
Abstract: On-policy distillation (OPD) transfers teacher capabilities by supervising trajectories sampled from the student's own policy, yet its generalizati...
By Zhaoyi Li, Deyang Kong, Yuan Wei, Evan Yang, Ranran Shen, Mahardika Krisna Ihsani, Ming Yang, Wei Zhang, Chuan Hao, Jian Yang, Ran Tao, Bryan Dai, Shikun Zhang, Wei Ye, Ying Wei, Defu Lian
On-policy distillation (OPD) transfers teacher capabilities by supervising trajectories sampled from the student's own policy, yet its generalization behavior remains poorly understood, as most studies evaluate OPD on a single domain and on benchmarks close to the training data. We present a controlled study that varies one generalization factor at a time, from in-domain distribution shifts to cross-domain transfer and the multi-teacher setting.
The paper introduces VISTA, a method that enhances on‑policy self‑distillation (OPSD) by adapting the teacher model toward the student’s distribution using outcome‑verified rollouts. VISTA keeps the standard OPSD student update but selectively adjusts the teacher only on the top‑k positions with the largest teacher‑student KL divergence, without adding new sampling or reward objectives. Experiments on AIME24, AIME25, and HMMT25 with Qwen3 models show that VISTA outperforms OPSD across all scales, improving Avg@12 by up to 2.1 points.
By Zewen Ding, Zezhong Wu, Zhou Tao, Shida Wang, Shizhuo Hou, YongXiang Hua, Haoyu Cao, Linli Xu