When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better

The paper investigates whether on‑policy distillation (OPD) is always advantageous for transferring knowledge from a teacher to a student model. It introduces Semi‑OPD, which uses offline rollouts from the initial student, and demonstrates that across 17 teacher‑student pairs (1.5B–235B parameters) Semi‑OPD outperforms OPD in 14 cases, achieving up to +13.6% accuracy and an 11.4× speedup. The authors show that the benefit of OPD depends on the alignment between teacher and student, measured by an output‑token overlap ratio, and argue that effective distillation requires on‑policy behavior relative to both models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• efficiency
• safety