← Back to all news
arXiv Computer Vision

CA-OPD: Confidence-Aware On-Policy Distillation for Structured Visual Prediction

CA-OPD is a confidence‑aware on‑policy distillation framework that improves structured visual prediction by using teacher confidence to selectively correct unreliable student transitions and gradually transfer rollout control to the student. The method aligns supervision with intervention decisions, providing direct cross‑entropy loss for corrected tokens and full predictive distribution for retained tokens. In a multi‑teacher setting for GUI grounding and OCR, CA‑OPD significantly outperforms the Qwen3.5‑0.8B baseline, achieving large gains on benchmarks such as ScreenSpot‑Pro and OCRBench‑v2 English.

Related stories

arXiv Machine Learning
Oct 16

Do As I Do, Not As I Say: On-Policy Distillation Inherits the Teacher in Vision-Language Models

The paper investigates how on‑policy distillation (OPD) inherits a teacher’s behavior in vision‑language models, comparing it to filtered supervised fine‑tuning (SFT) across different data budgets and tasks. Experiments on ChartQA and Geometry3K show that OPD performs well only when the teacher remains reliable on student‑generated trajectories; otherwise it underperforms SFT and can erase previously learned completion behavior. Filtering teacher demonstrations for completeness largely restores SFT’s advantages, indicating that OPD can transfer both knowledge and persistent failure modes from the teacher.

By Frank F. Yang, Kefei Yan, Yiren Wang, Yiyang Wang
llmsfine-tuningefficiencymultimodal
More like this →
arXiv Machine Learning
Sep 1

Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement

The paper investigates on‑policy distillation (OPD), showing that teacher supervision during OPD contains significant noise that grows with teacher size, yet the student policy remains largely unaffected by this noise. It finds that OPD’s gains stem mainly from suppressing low‑log‑probability tokens, a process that can be replicated without a teacher. Building on this insight, the authors propose On‑Policy Self‑Adaptation (OPSA), a supervision‑free method that uses entropy‑adaptive negative advantages to improve performance on several benchmarks, outperforming both the base model and OPD.

By Yi Ding, Ruqi Zhang
reinforcement-learningefficiencybenchmarks
More like this →
arXiv AI
Aug 18

Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuning

arXiv:2608. 16333v1 Announce Type: cross Abstract: On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories.

By Changhui Sun, Lanbo Liu, Hang Lei, Tong Ling, Jiahang Xie, Zhiyong Zheng, Yujia Wang, Hao Liu, Feng Xiao, Lu Liu, Yanlong Du, Zifeng Cheng, Ziwei Jiang, Qing Gu
agentsfine-tuningefficiency
More like this →
arXiv Machine Learning
Oct 2

Activation-Conditioned Self-Distillation

Activation-Conditioned Self-Distillation (ACSD) is a new on‑policy self‑distillation method that uses a frozen copy of the base model to extract a steering vector by contrasting activations from self‑generated trajectories that reach verified correct answers with all other trajectories. The student learns from next‑token distributions on its own prefixes, without needing reference text or teacher parameter updates, and is used alone at inference. Across five models, ACSD achieves the highest mean accuracy on four mathematical benchmarks, with notable gains on DeepSeek‑R1‑0528‑Qwen3‑8B and LiveCodeBench v6 compared to the OPSD baseline.

By Zhexi Lu, Subhajit Chaudhury, Tejaswini Pedapati, Keerthiram Murugesan, Lei Yu
efficiencybenchmarks
More like this →