← Back to all news
arXiv AI By Yun Zhu, Da Wang

TAD: Distilling Vision-Language-Action Policies from Teacher Trajectories Alone

Read the original on arXiv AI →

The paper introduces Trajectory-only Adversarial Distillation (TAD), a method that distills vision‑language‑action policies using only teacher trajectory data without requiring teacher action scores. TAD employs a discriminator to compare feature marginals from teacher and student rollouts, providing adaptive rewards for on‑policy learning. In experiments on the ManiSkill task PutOnPlateInScene25Main, TAD significantly boosts OpenVLA‑OFT performance from 26.2% to 76.6% success‑once and from 20.3% to 58.6% success‑end, approaching white‑box on‑policy distillation while using fewer teacher queries and interaction budgets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 16

Do As I Do, Not As I Say: On-Policy Distillation Inherits the Teacher in Vision-Language Models

The paper investigates how on‑policy distillation (OPD) inherits a teacher’s behavior in vision‑language models, comparing it to filtered supervised fine‑tuning (SFT) across different data budgets and tasks. Experiments on ChartQA and Geometry3K show that OPD performs well only when the teacher remains reliable on student‑generated trajectories; otherwise it underperforms SFT and can erase previously learned completion behavior. Filtering teacher demonstrations for completeness largely restores SFT’s advantages, indicating that OPD can transfer both knowledge and persistent failure modes from the teacher.

By Frank F. Yang, Kefei Yan, Yiren Wang, Yiyang Wang
llmsfine-tuningefficiencymultimodal
More like this →
arXiv Machine Learning
Oct 23

When to Follow the Teacher: Advantage-Based On-Policy Distillation

The paper introduces Advantage-Based On-Policy Distillation (AOPD), a method that selectively weights teacher actions during online policy distillation by their counterfactual advantage to the student. Unlike standard OPD, which follows the teacher at every state, AOPD reinforces only those teacher actions that improve the student's expected outcome, using an estimator that relies solely on standard student rollouts and terminal rewards. Experiments on math, coding, and Medqa tasks demonstrate that AOPD outperforms both distillation and reinforcement learning baselines in external-teacher and self-distillation settings.

By Mahdi Sabbaghi, George Pappas, Hamed Hassani, Adel Javanmard
reinforcement-learningfine-tuningefficiency
More like this →
arXiv Machine Learning
Sep 1

Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement

The paper investigates on‑policy distillation (OPD), showing that teacher supervision during OPD contains significant noise that grows with teacher size, yet the student policy remains largely unaffected by this noise. It finds that OPD’s gains stem mainly from suppressing low‑log‑probability tokens, a process that can be replicated without a teacher. Building on this insight, the authors propose On‑Policy Self‑Adaptation (OPSA), a supervision‑free method that uses entropy‑adaptive negative advantages to improve performance on several benchmarks, outperforming both the base model and OPD.

By Yi Ding, Ruqi Zhang
reinforcement-learningefficiencybenchmarks
More like this →