arXiv:2608. 09447v1 Announce Type: cross Abstract: On-policy distillation (OPD) aligns a student with a teacher on trajectories sampled from the student itself, reducing the train-test state mismatch of offline distillation.
By Zehao Chen, Gongxun Li, Tianxiang Ai, Yifei Li, Zixuan Huang, Wang Zhou, Tao Huang, Fuzhen Zhuang, Xianglong Liu, Jianxin Li, Deqing Wang, Yikun Ban
The paper introduces Group‑Calibrated On‑Policy Distillation (GC‑OPD), a method that aligns token‑level teacher guidance with trajectory‑level verifier rewards for long‑context reasoning tasks. GC‑OPD normalizes rewards within rollout groups, uses the signed teacher‑verifier disagreement as a residual, and distributes this residual across tokens via Relative‑Advantage‑Based Credit Assignment (RACA). Experiments on five long‑context benchmarks show that GC‑OPD improves Qwen3‑4B and Qwen3‑8B checkpoints from 29.08/35.12 to 40.47/44.65, outperforming vanilla OPD and demonstrating the effectiveness of group‑relative residual calibration.
By Zhu Zhang, Jixun Wang, Xiaoang Xu, Xiaorong Wang, Zihan Zhou, Zhiyuan Wang, Shuo Wang, Chaojun Xiao, Yuezhi Zhou
The paper introduces Group-Calibrated On-Policy Distillation (GC‑OPD), a method that aligns token‑level teacher guidance with task‑level verifier rewards for long‑context reasoning. GC‑OPD normalizes verifier and OPD scores within rollout groups, uses their difference as a signed disagreement residual, and redistributes this residual across tokens via Relative‑Advantage‑Based Credit Assignment (RACA). Experiments on five long‑context benchmarks show that GC‑OPD improves Qwen3‑4B and Qwen3‑8B checkpoints from 29.08/35.12 to 40.47/44.65, outperforming vanilla OPD and demonstrating the effectiveness of group‑relative residual calibration.
arXiv:2608. 07935v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) adapts a language model by distilling guidance from a frozen teacher on trajectories sampled from the student.
By Meilin Yang (Renmin University of China, Beijing, China), Zixuan Ding (Renmin University of China, Beijing, China), Jianhao Nie (Renmin University of China, Beijing, China), Weite Zhang (Renmin University of China, Beijing, China), Yuxin Zhang (Renmin University of China, Beijing, China), Zhiming Shao (Renmin University of China, Beijing, China), Li Yu (Renmin University of China, Beijing, China), Zhe Fu (Renmin University of China, Beijing, China)
arXiv:2609.14636v1 Announce Type: new
Abstract: On-policy distillation (OPD) has become a standard approach for transferring capabilities from large teachers to compact students. Its cost, however, i...
By Zhiyu Gui, Kexin Huang, Jia Guo, Junkang Wu, Zihao Wang, Zhiqiang Zhang, Jun Zhou, Jiancan Wu, Xiang Wang
CA-OPD is a confidence‑aware on‑policy distillation framework that improves structured visual prediction by using teacher confidence to selectively correct unreliable student transitions and gradually transfer rollout control to the student. The method aligns supervision with intervention decisions, providing direct cross‑entropy loss for corrected tokens and full predictive distribution for retained tokens. In a multi‑teacher setting for GUI grounding and OCR, CA‑OPD significantly outperforms the Qwen3.5‑0.8B baseline, achieving large gains on benchmarks such as ScreenSpot‑Pro and OCRBench‑v2 English.
By Menghao Li, Linjie Mu, Yin Wang, Haotian Hu, Yannian Gu, Lujiayi Xue, Fanyi Wang