arXiv Machine Learning
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Aug 11
PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation
arXiv:2608. 08726v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) uses a privileged teacher to supervise a reasoning model on prefixes sampled from its own rollouts.
arXiv AI
Aug 11
Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation
arXiv:2608. 09228v1 Announce Type: cross Abstract: On-Policy Self-Distillation (OPSD) is commonly interpreted as the transfer of privileged information: a teacher observes the verified solution to the target problem and supervises the student's trajectory.
arXiv Machine Learning
Sep 23
What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation
arXiv:2609.25623v1 Announce Type: new Abstract: More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a frozen copy of...
arXiv Computation and Language
Sep 18
What Does Privileged Information Add to On-Policy Self-Distillation?
The paper investigates how privileged information—such as a teacher’s full solution or reasoning trace—affects on‑policy self‑distillation (OPSD) in language models. Using the AMPLE‑Math benchmark, the authors compare distillation with and without extra teacher views, finding that reference‑free distillation explains most gains for Qwen3‑1.7B, while additional references provide modest benefits, especially for polished solutions. The study also shows that the impact of privileged data depends on the student’s training regime and that altering token‑level supervision can leave student behavior largely unchanged.
Hugging Face Trending Papers
Aug 26
One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation
On-policy distillation trains a language model on its own generations while a teacher scores them token by token. It combines the dense supervision of imitation learning with the on-policy sampling of...
arXiv Machine Learning
Aug 27
One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation
The paper reviews On‑Policy Self‑Distillation (OPSD), a method where a language model learns from its own generations using privileged information such as reference solutions or plans, eliminating the need for a larger teacher model. It identifies a key failure mode—collapse, where the model’s reasoning paths narrow progressively—and analyzes it through three levers: signal application, privileged information, and teacher dynamics. The review focuses on mathematical reasoning, offering a unified vocabulary and distinguishing settled facts from ongoing debates.