arXiv AI
Jul 21
Trace-Based On-Policy Distillation for Masked Diffusion Language Models
arXiv:2607. 16872v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) are a promising alternative to autoregressive generation.
dOPSD: On-Policy Self-Distillation for Diffusion Language Models
arXiv:2607. 04428v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods.
Related stories
arXiv Machine Learning
Jul 22
H$^2$SD: Hybrid Hindsight Self-Distillation
arXiv:2607. 18955v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models on tasks such as mathematical reasoning and code generation.
arXiv AI
Jul 3
DemoPSD: Disagreement-Modulated Policy Self-Distillation
arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.
arXiv AI
Aug 6
Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation
arXiv:2608. 04794v1 Announce Type: new Abstract: Self-distillation (SD) has emerged as a compute-efficient alternative to reinforcement learning with verifiable rewards: a self-teacher, conditioned on privileged information (PI) about the answer such as a reference solution, supplies dense per-token supervision to a student that never sees it.
arXiv AI
Jul 3
Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
arXiv:2607. 02234v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) has emerged as a promising paradigm for improving LLM reasoning, where a privileged teacher with access to reference solutions provides token-level supervision on the student's own generated trajectories.
arXiv AI
Aug 5
When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation
arXiv:2608. 03632v1 Announce Type: new Abstract: On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals.
arXiv AI
Aug 11
Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation
arXiv:2608. 08176v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) improves the reasoning abilities of LLMs by internalizing privileged context into model parameters through self-distillation.
arXiv AI
Aug 7
DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models
arXiv:2608. 06243v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models using automatically verifiable outcome signals, but these signals are typically sparse and at the sequence-level.
arXiv AI
Aug 11
SR-OPSD: Self-Referenced On-Policy Self-Distillation
arXiv:2608. 09745v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) converts feedback into dense token-level supervision on trajectories generated by the policy to be optimized, providing a useful complement to reinforcement learning with sparse outcome rewards.
Hugging Face Trending Papers
Jul 6
Weak-to-Strong Generalization via Direct On-Policy Distillation
Reinforcement learning with verifiable rewards (RLVR) is a powerful recipe for improving language-model reasoning, but it is expensive to repeat on every new strong model because the target model must generate many rollouts during training. As models scale, post-training itself becomes a bottleneck.
arXiv Machine Learning
Jun 11
RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation
arXiv:2606. 11709v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) provides dense, token-level supervision for reasoning models by aligning a model's own distribution with the distribution it produces under privileged context, typically a verified solution.
arXiv AI
Jun 9
Trajectory-Refined Distillation
arXiv:2606. 08432v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a central post-training tool for large language models (LLMs), providing dense per-token teacher supervision along the student's own rollouts.