arXiv:2605. 03677v2 Announce Type: replace Abstract: On-policy distillation (OPD) has recently emerged as an effective post-training paradigm for consolidating the capabilities of specialized expert models into a single student model.
The paper investigates self‑distillation techniques for language models by systematically varying three key design choices: the source of rollout tokens (student vs. teacher), the teacher coupling strategy (frozen or exponential moving average), and the KL divergence direction (reverse or forward). Experiments on Qwen2.5‑7B and Ministral‑3‑3B across 1,200 adaptation runs reveal that rollout source mainly affects acquisition on contradictory tasks, teacher coupling most strongly influences acquisition across all tasks, and KL direction impacts retention differently depending on the model. A controlled theoretical model reproduces these empirical trends, offering a unified framework for understanding acquisition‑retention trade‑offs in self‑distillation.
arXiv:2607. 17558v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) offers a promising approach for training large language models without relying on a separate teacher model.
The paper investigates whether a strong teacher is necessary for knowledge distillation in large language model pretraining. By varying teacher and student sizes and token budgets, the authors create strong‑to‑weak, same‑level, and weak‑to‑strong teacher‑student pairs and evaluate distillation effectiveness. They find that even small, undertrained teachers can improve larger students when losses are properly mixed, and that pushing teachers to be stronger can saturate or reverse gains, while distillation mainly boosts generalization rather than in‑domain fitting.
arXiv:2608. 07935v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) adapts a language model by distilling guidance from a frozen teacher on trajectories sampled from the student.
arXiv:2608.22898v1 Announce Type: new Abstract: Diffusion language models (DLMs) alleviate the inherent latency bottleneck of autoregressive (AR) large language models (LLMs), but their degraded gene...
arXiv:2606. 08432v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a central post-training tool for large language models (LLMs), providing dense per-token teacher supervision along the student's own rollouts.
Activation-Conditioned Self-Distillation (ACSD) is a new on‑policy self‑distillation method that uses a frozen copy of the base model to extract a steering vector by contrasting activations from self‑generated trajectories that reach verified correct answers with all other trajectories. The student learns from next‑token distributions on its own prefixes, without needing reference text or teacher parameter updates, and is used alone at inference. Across five models, ACSD achieves the highest mean accuracy on four mathematical benchmarks, with notable gains on DeepSeek‑R1‑0528‑Qwen3‑8B and LiveCodeBench v6 compared to the OPSD baseline.
arXiv:2608. 06296v1 Announce Type: new Abstract: On-policy (Self-)Distillation (OPD / OPSD) has shown strong potential for post-training large language models (LLMs).
On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals. Recent selective OPD methods improve this process by prioritizing signals that are confident, informative, or learnable.
The paper introduces Self-Regularized Flow Language Models (SERF), a self‑supervised framework that enhances continuous diffusion language models by leveraging the denoising trajectory. SERF employs noisy‑to‑clean self‑distillation, where an exponential moving average teacher provides additional supervision along the same interpolation path, and offers two variants—SERF‑L aligning intermediate representations and SERF‑P aligning output token distributions. Experiments on three continuous language‑model paradigms show significant gains in training speed, generation quality, and sampling efficiency, achieving up to 7.7× faster training, 24.3% MAUVE improvement, and 28.8% higher GSM8K accuracy. "whyItMatters":"The results demonstrate that noisy‑to‑clean self‑distillation can substantially improve continuous diffusion language models without external pretraining or inference changes, offering a practical path to faster, higher‑quality language generation."
arXiv:2605.10889v2 Announce Type: replace-cross Abstract: On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions...