The paper introduces On-Policy Mid-Training Distillation (OPMD), a method that shifts continual pre‑training from passive next‑token prediction to active, query‑driven knowledge acquisition. By sampling trajectories from the evolving student policy and applying dense token‑level reverse‑KL rewards from a frozen teacher, OPMD trains the model on states it actually visits, reducing the mismatch between training and generation. Experiments on MATH‑500 and Minerva show that OPMD improves a Qwen student’s performance over its base and outperforms the standard next‑token prediction baseline, while integrating seamlessly into existing continual pre‑training pipelines without inference‑time overhead.
By Ruize Zheng, Xuyang Liu, Siyu Ren, Xiaobing Tu, Yinggui Wang, Jinkui Ren, Xiantao Zhang, Linfeng Zhang
arXiv:2606. 06712v1 Announce Type: cross Abstract: We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs).
By Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi, Lakshmi Jotsna, Degui Zhi, James Caverlee, Dileep Kalathil, Shuiwang Ji
arXiv:2610.19601v1 Announce Type: new
Abstract: Knowledge distillation (KD) improves language-model pretraining by matching teacher distributions at corpus prefixes, but provides no direct supervisio...
By Zhenhe Wu, Jian Yang, Yaping Jin, Xinlai Kang, Jie Hu, Hanting Chen
arXiv:2606. 09456v1 Announce Type: new Abstract: On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models.
By Yifan Niu, Han Xiao, Dongyi Liu, Zelong Wang, Dihong Gong, Yasheng Wang, Jia Li
Knowledge distillation (KD) improves language-model pretraining by matching teacher distributions at corpus prefixes, but provides no direct supervision at contexts induced by student predictions. We...
arXiv:2608.28647v1 Announce Type: new
Abstract: Target-only post-training can improve performance in a specialized domain while degrading behaviors that a general-purpose base model acquired before a...
By Yifei Li, Rongman Xu, Lingling Zhang, Muye Huang, Zihan Ma, Jiashuai Liu, Hang Yan, Heng Wang