← Back to all news
arXiv Machine Learning By Jie Yang, Zhengyu Fang, Zelin Xu, Jiarui Sun, Xiran Fan, Junpeng Wang, Liang Wang, Qinghua Liu, Yiwei Cai, Yan Zheng

LastOPD: Taming Collapse in Latent On-Policy Distillation

Read the original on arXiv Machine Learning →

The paper introduces LastOPD, a method that mitigates collapse in latent on‑policy distillation by applying latent supervision only to the last‑layer state during a brief cross‑fade into token‑level OPD. Experiments show that LastOPD improves MATH‑500 accuracy by 5.55 and 4.02 points over token‑only OPD when distilling Qwen3‑4B and Qwen3‑8B into Qwen3‑1.7B‑Base, and achieves comparable final scores in roughly half the training steps.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Aug 11

Mismatch Matters: On-Policy Distillation Beyond Token Agreement

arXiv:2608. 09836v1 Announce Type: new Abstract: On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses.

By Zichao Yu, Chengzhi Yu, Shengze Xu, Yujin Han, Bingqing Jiang, Xu Wang, Difan Zou
llmsefficiencybenchmarks
More like this →
arXiv AI
Sep 25

Not Every Token Is Worth Distilling: Selective Supervision for Direct-OPD

The paper introduces Selective Supervision for Direct-OPD (S$^2$D-OPD), a refinement of Direct On-Policy Distillation that filters out states where the teacher’s policy change is minimal, as measured by the teacher‑reference Jensen‑Shannon divergence. By masking low‑divergence states and keeping only the top 10% of states per response, S$^2$D-OPD improves held‑out accuracy on AIME and HMMT benchmarks across multiple teacher‑student pairs without additional forward passes.

By Yibo Zhao, Zixuan Yang, Yunshi Lan, Xiang Li
reinforcement-learningefficiencybenchmarks
More like this →
arXiv Machine Learning
Jun 5

OPRD: On-Policy Representation Distillation

arXiv:2606. 06021v1 Announce Type: new Abstract: On-policy distillation (OPD) supervises the student only in output space by matching next-token probabilities.

By Shenzhi Yang, Guangcheng Zhu, Bowen Song, Haobo Wang, Mingxuan Xia, Xing Zheng, Yingfan Ma, Zhongqi Chen, Weiqiang Wang, Gang Chen
efficiency
More like this →
arXiv AI
Jun 9

Trajectory-Refined Distillation

arXiv:2606. 08432v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a central post-training tool for large language models (LLMs), providing dense per-token teacher supervision along the student's own rollouts.

By Li Jiang, Haoran Xu, Yichuan Ding, Amy Zhang
llmsefficiencybenchmarks
More like this →