arXiv AI
Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 28
Masked Distillation: Internalizing the Chain-of-Thought in Language Models
arXiv:2607. 22629v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) produce long, explicit chains of intermediate steps before generating a final answer at inference time.
arXiv AI
Jul 3
Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
arXiv:2607. 02234v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) has emerged as a promising paradigm for improving LLM reasoning, where a privileged teacher with access to reference solutions provides token-level supervision on the student's own generated trajectories.
arXiv AI
Aug 11
DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation
arXiv:2605. 15532v3 Announce Type: replace-cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics or aggregated from off-the-shelf datasets.
arXiv AI
2d ago
STEPS: Selective On-Policy Self-Distillation for Reasoning
arXiv:2605.10194v2 Announce Type: replace Abstract: On-policy self-distillation (OPSD) uses a model as its own teacher under privileged context, providing token-level supervision on the model's own r...
arXiv AI
Jul 7
Rethinking On-Policy Self-Distillation for Thinking Models
arXiv:2607. 05184v1 Announce Type: new Abstract: Self-distillation is a promising recipe for self-improvement in language models.
arXiv AI
Sep 10
Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
arXiv:2605.10889v2 Announce Type: replace-cross Abstract: On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions...