Hugging Face Trending Papers
Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Oct 21
Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
arXiv:2610.27450v1 Announce Type: new Abstract: On-policy distillation (OPD) can succeed with surprisingly few training questions, motivating a view of questions as carriers of distributional knowled...
arXiv AI
Oct 9
Data-Free On-Policy Distillation: How Far Can We Go Without External Data?
The paper investigates how much training data is required for on‑policy distillation (OPD) of foundation models. It shows that in two single‑teacher settings, only eight real prompts can match the performance of 17,000 problems, and that 64 self‑generated questions can replace full external datasets. In multi‑teacher scenarios, 1,000 generated questions achieve comparable results to 7,000 real examples, while fully implicit question generation works only in limited cases.
arXiv AI
Oct 28
Proposing Better Rollouts for On-Policy Distillation
The paper proposes Mixture Proposal On-Policy Distillation (MPOPD) to address a mismatch issue in on‑policy distillation (OPD) where the rollout policy is fixed to the current student. By treating the rollout policy as an optimization target, MPOPD derives a geometric mixture of student and teacher policies and introduces a proposal policy for efficient rollout generation. Experiments on 16 reasoning benchmarks show consistent gains over OPD, notably a 6.91% accuracy improvement on math reasoning with Qwen3-4B-Base and a 41% reduction in generation length.
arXiv AI
Jul 3
DemoPSD: Disagreement-Modulated Policy Self-Distillation
arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.
arXiv AI
Sep 15
Data-free On-policy Distillation
arXiv:2609.14193v1 Announce Type: cross Abstract: On-policy distillation (OPD) has become a standard component of frontier post-training pipelines, yet how much its training data actually contributes...
arXiv Computation and Language
Sep 28
Recursive Self-Improvement via On-Policy Distillation for Reasoning
The paper introduces a recursive self-improvement framework for language models that replaces an external teacher with a frozen copy of the student, enabling dynamic co-evolution (DCE) and self-refined concise learning (SRCL). DCE allows the privileged teacher to evolve alongside the student, while SRCL trains on shorter, verified rewrites to reduce verbosity. Experiments show that the combined DCE+SRCL approach outperforms traditional on‑policy self‑distillation across multiple model sizes and math benchmarks, achieving significant accuracy gains and shorter outputs.