SimCT is a method for on‑policy distillation that addresses the problem of mismatched tokenizers between teacher and student models. By comparing teacher and student predictions over short multi‑token continuations that both tokenizers can produce, SimCT restores supervision that would otherwise be lost when only exact shared tokens are used. Experiments on mathematical reasoning and code‑generation tasks show consistent improvements over shared‑vocabulary OPD and other cross‑tokenizer baselines, with ablations confirming the gains come from recovering discarded supervision.
arXiv:2610.08448v1 Announce Type: cross Abstract: On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student...
arXiv:2608. 14277v1 Announce Type: cross Abstract: On-policy distillation (OPD) offers a promising way to transfer reasoning capabilities from stronger teacher models, but applying it to long-context reasoning teachers and short-context students introduces practical challenges, including tokenizer mismatch, teacher-student distribution mismatch, response length explosion, and training instability.
arXiv:2607. 22334v1 Announce Type: new Abstract: Open-weight language models from different families exhibit complementary capabilities, motivating their consolidation into a compact student through on-policy distillation (OPD).
The paper investigates on‑policy distillation (OPD) for large language models, showing that OPD learns not only which tokens to consider but also where to keep reasoning paths open. It introduces the entropy shape—a sequence of next‑token entropies—to capture this "where to branch" aspect and proposes the Entropy Shape Correlation (ESC) metric to measure student‑teacher agreement. By tracking ESC during training and adjusting a teacher’s entropy while preserving token preferences, the authors demonstrate that controlling where to branch causally drives OPD gains, and they establish a teacher‑selection protocol where pre‑OPD ESC predicts the best teacher more accurately than existing methods.
arXiv:2610.14022v1 Announce Type: cross Abstract: On-policy distillation (OPD) improves language-model by supervising a student on prefixes sampled from its own policy, but its supervision remains co...
arXiv:2606. 08432v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a central post-training tool for large language models (LLMs), providing dense per-token teacher supervision along the student's own rollouts.
CAST (Chunk-aware credit Assignment with Selective Token Anchoring) is a new framework for cross-tokenizer on-policy distillation that tackles two key problems: credit alignment and unaligned token positions. It redistributes chunk-level advantage signals across student tokens using predictive uncertainty and teacher-signal smoothness, while anchoring unaligned tokens to the initial policy via a KL penalty. Experiments on six mathematics, science, and code benchmarks show that CAST improves average performance over the strongest baseline by up to 2.45 points in cross-tokenizer transfer scenarios.
arXiv:2607. 04037v1 Announce Type: cross Abstract: On-policy distillation is a powerful way to transfer reasoning ability from a strong teacher to a smaller student: the student samples trajectories from its own policy, and the teacher provides dense token-level supervision on the states the student actually visits.
arXiv:2606. 09456v1 Announce Type: new Abstract: On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models.
arXiv:2609.33455v2 Announce Type: replace Abstract: On-policy distillation (OPD) trains a student model on its own trajectories using dense token-level feedback from a stronger teacher model. Since e...
The paper introduces Common‑Support Distillation (CSD), a logits‑based framework for transferring knowledge across different tokenizers by constructing a shared support between teacher and student distributions. CSD handles both one‑to‑one alignments using explicit common‑token pairs and other alignments by creating a virtual support via the autoregressive chain rule, while a residual event ensures normalization. Experiments on Llama‑3.2‑1B, Gemma3‑1B‑PT, Qwen, and Phi models show that CSD consistently outperforms existing cross‑tokenizer methods, achieving up to 1.8‑point gains over the best alternative and surpassing both continued pretraining and same‑tokenizer distillation.
SOD: Step-wise On-policy Distillation for Small Language Model Agents proposes a new framework that adaptively reweights distillation strength at each reasoning step based on step-level divergence. This approach mitigates cascading errors in tool-integrated reasoning by attenuating misleading teacher signals in high-divergence regions while preserving dense guidance where student and teacher align. Experiments on math, science, and code benchmarks show up to 20.86% improvement over the second-best baseline, with a 0.6B student scoring 26.13% on AIME 2025.