arXiv AI
Jul 7

Reward-Gated On-Policy Distillation

arXiv:2607. 04037v1 Announce Type: cross Abstract: On-policy distillation is a powerful way to transfer reasoning ability from a strong teacher to a smaller student: the student samples trajectories from its own policy, and the teacher provides dense token-level supervision on the states the student actually visits.

By Mohammad Sadegh Akhondzadeh, Vijay Lingam, Atula Tejaswi, Chanakya Ekbote, Sujay Sanghavi, Aleksandar Bojchevski
arXiv Machine Learning
Nov 6

Reinforcing What the Student Has Learned: Teacher-Guided On-Policy Distillation

The paper introduces Student-Delta On-Policy Distillation (SD-OPD), a method that refines language-model reasoning by using the student’s own pretrained base as a reference for teacher feedback. Unlike traditional on‑policy distillation (OPD), SD‑OPD scales updates based on centered student‑base differences, applies them only when they align with teacher‑student feedback, and recomputes the student signal each training round. Experiments with Qwen3 students ranging from 1.7B to 8B parameters show that SD‑OPD boosts six‑task mathematics average pass@1 by 7.76 to 10.37 percentage points over OPD.

By Bozhen Fang, Linhao Zhong, Xuehao Wang, Huanyi Zheng, Hao Chen, Chunhua Shen
arXiv AI
Aug 11

Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation

arXiv:2608. 08176v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) improves the reasoning abilities of LLMs by internalizing privileged context into model parameters through self-distillation.

By Yongkang Yang, Zhezheng Hao, Hong Zhang, Yi Liu, Xiankun Lin, Wence Ji, Fanjunduo Wei, Jiarui Yu, Qiang Lin, Xiaoyun Liang, Hande Dong