The paper introduces Common‑Support Distillation (CSD), a logits‑based framework for transferring knowledge across different tokenizers by constructing a shared support between teacher and student distributions. CSD handles both one‑to‑one alignments using explicit common‑token pairs and other alignments by creating a virtual support via the autoregressive chain rule, while a residual event ensures normalization. Experiments on Llama‑3.2‑1B, Gemma3‑1B‑PT, Qwen, and Phi models show that CSD consistently outperforms existing cross‑tokenizer methods, achieving up to 1.8‑point gains over the best alternative and surpassing both continued pretraining and same‑tokenizer distillation.
By Mingyu Yang, Sharath Turuvekere Sreenivas, Adithyakrishna Venkatesh Hanasoge, Nima Tajbakhsh, Ashwath Aithal, Pavlo Molchanov
arXiv:2606. 08432v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a central post-training tool for large language models (LLMs), providing dense per-token teacher supervision along the student's own rollouts.
By Li Jiang, Haoran Xu, Yichuan Ding, Amy Zhang
arXiv:2608. 03632v1 Announce Type: new Abstract: On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals.
By Yinuo Jiang, Yongjie Ye, Zhou Tao, Xiang Zhuang, Qiang Zhang, Huajun Chen, Tiankai Li
arXiv:2605. 03677v2 Announce Type: replace Abstract: On-policy distillation (OPD) has recently emerged as an effective post-training paradigm for consolidating the capabilities of specialized expert models into a single student model.
By Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, Chengquan Zhang, Zhuotao Tian, Han Hu, Yi Yang, Fei Wu, Hehe Fan
arXiv:2601. 07155v3 Announce Type: replace-cross Abstract: Knowledge distillation (KD) is a widely adopted technique for transferring knowledge from large language models to smaller student models; however, conventional supervised KD often suffers from a distribution mismatch between training and inference.
By Ijun Jang, Jewon Yeom, Juan Yeo, Hyunggyu Lim, Taesup Kim
On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals. Recent selective OPD methods improve this process by prioritizing signals that are confident, informative, or learnable.
The paper investigates on‑policy distillation (OPD) for large language models, showing that OPD learns not only which tokens to consider but also where to keep reasoning paths open. It introduces the entropy shape—a sequence of next‑token entropies—to capture this "where to branch" aspect and proposes the Entropy Shape Correlation (ESC) metric to measure student‑teacher agreement. By tracking ESC during training and adjusting a teacher’s entropy while preserving token preferences, the authors demonstrate that controlling where to branch causally drives OPD gains, and they establish a teacher‑selection protocol where pre‑OPD ESC predicts the best teacher more accurately than existing methods.
By Shiping Yang, Chuanqi Tang, Hossein Hajimirsadeghi, Layla El Asri, Mohammad Amin Shabani, Angel X. Chang
arXiv:2605.10889v2 Announce Type: replace-cross
Abstract: On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions...
By Mohammadreza Armandpour, Fatih Ilhan, David Harrison, Ajay Jaiswal, Duc N. M Hoang, Fartash Faghri, Yizhe Zhang, Minsik Cho, Mehrdad Farajtabar
SimCT is a method for on‑policy distillation that addresses the problem of mismatched tokenizers between teacher and student models. By comparing teacher and student predictions over short multi‑token continuations that both tokenizers can produce, SimCT restores supervision that would otherwise be lost when only exact shared tokens are used. Experiments on mathematical reasoning and code‑generation tasks show consistent improvements over shared‑vocabulary OPD and other cross‑tokenizer baselines, with ablations confirming the gains come from recovering discarded supervision.
By Jie Sun, Mao Zheng, Mingyang Song, Qiyong Zhong, Yilin Cheng, Bichuan Feng, Pengfei Liu, Junfeng Fang, Xiang Wang
Activation-Conditioned Self-Distillation (ACSD) is a new on‑policy self‑distillation method that uses a frozen copy of the base model to extract a steering vector by contrasting activations from self‑generated trajectories that reach verified correct answers with all other trajectories. The student learns from next‑token distributions on its own prefixes, without needing reference text or teacher parameter updates, and is used alone at inference. Across five models, ACSD achieves the highest mean accuracy on four mathematical benchmarks, with notable gains on DeepSeek‑R1‑0528‑Qwen3‑8B and LiveCodeBench v6 compared to the OPSD baseline.
By Zhexi Lu, Subhajit Chaudhury, Tejaswini Pedapati, Keerthiram Murugesan, Lei Yu
arXiv:2606. 09456v1 Announce Type: new Abstract: On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models.
By Yifan Niu, Han Xiao, Dongyi Liu, Zelong Wang, Dihong Gong, Yasheng Wang, Jia Li
arXiv:2604. 10688v2 Announce Type: replace-cross Abstract: On-policy reinforcement learning has become the dominant paradigm for reasoning alignment in large language models, yet its sparse, outcome-level rewards make token-level credit assignment notoriously difficult.
By Binbin Zheng, Xing Ma, Yiheng Liang, Jingqing Ruan, Xiaoliang Fu, Kepeng Lin, Benchang Zhu, Ke Zeng, Xunliang Cai