arXiv Machine Learning
Jun 5
OPRD: On-Policy Representation Distillation
arXiv:2606. 06021v1 Announce Type: new Abstract: On-policy distillation (OPD) supervises the student only in output space by matching next-token probabilities.
PHF: Privileged Hidden Flow for On-Policy Self-Distillation
arXiv:2606. 29340v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a reasoning model on rollouts sampled from its own policy by matching a privileged teacher that also sees verified reference solutions.
Related stories
arXiv AI
Aug 11
SR-OPSD: Self-Referenced On-Policy Self-Distillation
arXiv:2608. 09745v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) converts feedback into dense token-level supervision on trajectories generated by the policy to be optimized, providing a useful complement to reinforcement learning with sparse outcome rewards.
arXiv AI
Jun 10
Beyond Absolute Imitation: Anchored Residual Guidance for Privileged On-Policy Distillation
arXiv:2606. 10385v1 Announce Type: cross Abstract: On-policy distillation (OPD) has demonstrated strong empirical gains in enhancing complex reasoning in LLMs by aligning a student model with a teacher's predictive distribution over the student's own trajectories.
arXiv AI
Aug 11
WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training
arXiv:2608. 09447v1 Announce Type: cross Abstract: On-policy distillation (OPD) aligns a student with a teacher on trajectories sampled from the student itself, reducing the train-test state mismatch of offline distillation.
arXiv AI
Aug 11
PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation
arXiv:2608. 08726v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) uses a privileged teacher to supervise a reasoning model on prefixes sampled from its own rollouts.
arXiv AI
Aug 7
When Privileged Guidance Misaligns: State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents
arXiv:2608. 05219v1 Announce Type: new Abstract: Privileged on-policy distillation provides dense supervision for multi-turn agents by allowing a synchronized teacher to re-score the student's response at every turn with access to training-only references, such as successful trajectories.
arXiv AI
Aug 7
DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models
arXiv:2608. 06243v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models using automatically verifiable outcome signals, but these signals are typically sparse and at the sequence-level.
arXiv Machine Learning
Aug 5
Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
arXiv:2608. 03316v1 Announce Type: new Abstract: On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid.
arXiv Machine Learning
Jul 16
Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
arXiv:2607. 13399v1 Announce Type: cross Abstract: On-policy distillation (OPD) has become a key paradigm in LLM post-training, yet its training dynamics remain poorly understood.
arXiv AI
Jul 7
Reward-Gated On-Policy Distillation
arXiv:2607. 04037v1 Announce Type: cross Abstract: On-policy distillation is a powerful way to transfer reasoning ability from a strong teacher to a smaller student: the student samples trajectories from its own policy, and the teacher provides dense token-level supervision on the states the student actually visits.
arXiv AI
Jul 31
ATOD: Annealed Turn-Aware On-Policy Distillation for Multi-Turn Agentic Tasks
arXiv:2606. 27814v4 Announce Type: replace Abstract: Training small language-model agents for long-horizon interactive tasks requires both fast imitation and reward-driven improvement.
arXiv AI
Jun 29
ATOD: Annealed Turn-aware On-policy Distillation for Multi-turn Autonomous Agents
arXiv:2606. 27814v1 Announce Type: new Abstract: Training small language-model agents for long-horizon interactive tasks requires both fast imitation and reward-driven improvement.