The paper introduces Localized Context Distillation (LoCD), a method that improves learning from interaction experience for large language models. LoCD identifies critical spans in a student’s trajectory that need revision, then has a teacher regenerate only those spans from the student’s own prefix, reducing exposure bias while focusing on informative states. Experiments across math reasoning, code generation, and agent tasks show that LoCD outperforms strong baselines and yields more stable training with smaller, sparser updates.
arXiv:2606. 11627v1 Announce Type: cross Abstract: Recent work has shown that on-policy distillation can internalize privileged context, such as system prompts or task hints, into a student model so that the context is no longer needed at inference time.
arXiv:2606. 08432v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a central post-training tool for large language models (LLMs), providing dense per-token teacher supervision along the student's own rollouts.
arXiv:2605.30251v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is rev...
arXiv:2606. 04703v1 Announce Type: cross Abstract: Experience internalization converts contextual experience from past interactions into reusable parametric capability, offering a promising path toward continual learning in large language models (LLMs).
The paper proposes a hierarchical framework for distilling large language model agents, distinguishing between the policy that generates interaction histories and the policy that produces token prefixes for actions. It explores various combinations of teacher-generated and student-generated data across multiple models and environments, finding that pairing student-induced histories with teacher-generated action prefixes yields the strongest training signal, while fully on-policy data is only beneficial when mixed with other configurations. The study also shows that the specific pairing of teacher and student proportions at each level significantly impacts performance, and that strategically front-loading teacher-generated contexts and delaying on-policy contexts improves results.
arXiv:2609.37500v1 Announce Type: new Abstract: On-policy distillation (OPD) trains language models using dense token-level teacher supervision on student-generated trajectories. However, its relianc...
arXiv:2610.14022v1 Announce Type: cross Abstract: On-policy distillation (OPD) improves language-model by supervising a student on prefixes sampled from its own policy, but its supervision remains co...
arXiv:2610.27847v1 Announce Type: new Abstract: Scaling the training of Large Language Model (LLM) agents typically requires a large collection of executable environments, whose construction, mainten...
arXiv:2607. 04763v1 Announce Type: cross Abstract: We study on-policy distillation (OPD) for agentic tasks, where an LLM agent interacts with an environment over multiple turns and a student imitates a teacher over these multi-turn interaction histories.
SOD: Step-wise On-policy Distillation for Small Language Model Agents proposes a new framework that adaptively reweights distillation strength at each reasoning step based on step-level divergence. This approach mitigates cascading errors in tool-integrated reasoning by attenuating misleading teacher signals in high-divergence regions while preserving dense guidance where student and teacher align. Experiments on math, science, and code benchmarks show up to 20.86% improvement over the second-best baseline, with a 0.6B student scoring 26.13% on AIME 2025.
arXiv:2608. 16333v1 Announce Type: cross Abstract: On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories.
The paper introduces Validity-Calibrated Reasoning Distillation, a new framework for transferring multi-step reasoning from large language models to smaller ones. Unlike prior methods that rely on static teacher‑student hierarchies and trajectory imitation, this approach treats distillation as a local learning‑signal allocation problem. By comparing the teacher’s and student’s next‑step actions under the same prefix and modulating updates based on their relative local validity, the method provides dynamic, context‑dependent supervision that preserves structural guidance while adapting to local reasoning quality. Experiments on math, code, and instruction‑following benchmarks show that this strategy outperforms strong distillation baselines.