arXiv:2608. 05219v1 Announce Type: new Abstract: Privileged on-policy distillation provides dense supervision for multi-turn agents by allowing a synchronized teacher to re-score the student's response at every turn with access to training-only references, such as successful trajectories.
arXiv:2606. 15912v1 Announce Type: cross Abstract: Multi-turn agents that plan, invoke tools, and interact with environments offer a promising paradigm for solving complex tasks, yet their capabilities typically rely on very large models whose inference cost is prohibitive in practice.
arXiv:2609.14636v1 Announce Type: new Abstract: On-policy distillation (OPD) has become a standard approach for transferring capabilities from large teachers to compact students. Its cost, however, i...
arXiv:2609. 40285v1 Announce Type: new Abstract: On-policy distillation (OPD) is a promising approach for training language agents, providing dense teacher supervision on student-generated trajectories.
arXiv:2609.37915v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies...
arXiv:2609.39687v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises studen...
arXiv:2608. 01953v1 Announce Type: cross Abstract: On-policy distillation (OPD) provides teacher supervision on states visited by the student, reducing the distribution gap between training and inference.
arXiv:2609.36608v1 Announce Type: new Abstract: On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-t...
ComputerSD is an online self‑distillation method for computer‑use agents that leverages real‑time feedback from executed GUI transitions. It uses a fine‑tuned GUI analyzer to generate guidance and a step‑level value score after each action, combining token‑level OPSD with trajectory‑level GRPO in an asynchronous training framework. On the OSWorld‑Verified benchmark, ComputerSD improves performance over outcome‑only GRPO by 1.9 and 4.1 percentage points on Qwen3‑VL‑8B‑Thinking and EvoCUA‑8B backbones, and shows strong generalizability in out‑of‑distribution tests.
arXiv:2607. 04763v1 Announce Type: cross Abstract: We study on-policy distillation (OPD) for agentic tasks, where an LLM agent interacts with an environment over multiple turns and a student imitates a teacher over these multi-turn interaction histories.
The paper introduces Privileged Self-Practice (PSP), a method that retains privileged information (PI) in the prompt rather than the loss during on‑policy self‑distillation for multi‑turn agents. PSP injects short per‑task instructions from an analyzer model when rollouts fail, sampling again with the instruction in context and training with the unchanged GRPO objective. Experiments on AppWorld and SWE‑bench Verified show PSP consistently outperforms plain GRPO, boosting task‑goal completion by up to 65% and resolved rate by up to 61% across three student models.
arXiv:2608. 16333v1 Announce Type: cross Abstract: On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories.
Online training enables computer-use agents (CUAs) to improve through interaction with executable environments. However, existing methods primarily rely on sparse outcome rewards, which provide no sup...