The Flow has not summarised this story yet — read it at arXiv Computation and Language.
On-policy distillation (OPD) supervises a student language model on trajectories sampled from its current policy, but assigns equal credit to response tokens with unequal supervision value. Selective OPD addresses this limitation by allocating supervision non-uniformly across response tokens according to their estimated training value.
arXiv:2608. 03632v1 Announce Type: new Abstract: On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals.
On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals. Recent selective OPD methods improve this process by prioritizing signals that are confident, informative, or learnable.
arXiv:2608. 07935v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) adapts a language model by distilling guidance from a frozen teacher on trajectories sampled from the student.
arXiv:2604. 10688v2 Announce Type: replace-cross Abstract: On-policy reinforcement learning has become the dominant paradigm for reasoning alignment in large language models, yet its sparse, outcome-level rewards make token-level credit assignment notoriously difficult.
CA-OPD is a confidence‑aware on‑policy distillation framework that improves structured visual prediction by using teacher confidence to selectively correct unreliable student transitions and gradually transfer rollout control to the student. The method aligns supervision with intervention decisions, providing direct cross‑entropy loss for corrected tokens and full predictive distribution for retained tokens. In a multi‑teacher setting for GUI grounding and OCR, CA‑OPD significantly outperforms the Qwen3.5‑0.8B baseline, achieving large gains on benchmarks such as ScreenSpot‑Pro and OCRBench‑v2 English.