arXiv Machine Learning
Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Oct 8
Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts
The paper introduces Δ‑MOPD, a multi‑teacher on‑policy distillation method that transfers each teacher’s logit shift relative to its base policy, re‑anchored at the student’s frozen initialization. Δ‑MOPD is evaluated in two settings: common‑domain composition, where multiple teachers score a single student rollout, and routed‑domain distillation, where prompts are assigned to specialist teachers. Across experiments, Δ‑MOPD outperforms traditional endpoint supervision, especially when teacher signals are combined at a state, achieving higher accuracy and reducing performance gaps in phased routing scenarios.
arXiv AI
Jun 16
On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
arXiv:2606. 15912v1 Announce Type: cross Abstract: Multi-turn agents that plan, invoke tools, and interact with environments offer a promising paradigm for solving complex tasks, yet their capabilities typically rely on very large models whose inference cost is prohibitive in practice.
Hugging Face Trending Papers
Sep 16
Trajectory Learnability for Offline On-Policy Distillation with Imperfect Teachers
Offline on-policy distillation gains efficiency by collecting student trajectories and teacher supervision once and reusing them throughout optimization. The same reuse makes imperfect supervision per...
arXiv AI
Sep 17
Trajectory Learnability for Offline On-Policy Distillation with Imperfect Teachers
The paper introduces a method for offline on‑policy distillation that addresses the problem of imperfect teacher supervision. By training on teacher‑successful problems and measuring changes in token likelihoods on teacher‑failed trajectories, the authors derive a learnability signal that weights the distillation loss. This approach improves performance on mathematical reasoning and code generation tasks while reducing computational cost compared to online distillation.
arXiv AI
Oct 21
Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy Learning
arXiv:2610.10460v2 Announce Type: replace-cross Abstract: Multi-teacher on-policy distillation (MOPD) is used in two settings. In common-domain composition, several teachers score each student rollou...
Hugging Face Trending Papers
Aug 17
Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuning
On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories. This approach has achieved strong empirical gains and can often surpass conve...