← Back to all news
arXiv Machine Learning By Haowei Wang, Hao Lin, Qi Zhu, Jiayi Ji, Deyi Ji, Liujuan Cao, Rongrong Ji

CV-OPD: Contrastive Vision On-policy Distillation

Read the original on arXiv Machine Learning →

The paper introduces Contrastive Vision On‑policy Distillation (CV‑OPD), a method that enhances on‑policy distillation for vision‑language models by adding a contrastive divergence loss and a discrepancy‑aware gating mechanism. CV‑OPD contrasts teacher distributions from positive and negative visual crops, forcing the student to rely on genuine visual evidence rather than shortcuts. Experiments on fine‑grained visual benchmarks show that CV‑OPD reduces shortcut learning and improves visual reasoning across existing VLMs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Aug 17

Self-Supervised Visual On-Policy Distillation

arXiv:2608. 14144v1 Announce Type: cross Abstract: Visual on-policy distillation relies heavily on an informative teacher-student asymmetry, through either a larger, stronger teacher or privileged supervision, such as reference answers or ground-truth regions of interest.

By Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr, Nuno Vasconcelos
llmsefficiencybenchmarks
More like this →
arXiv AI
Jul 24

Visual Contrastive Self-Distillation

arXiv:2607. 21556v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) is promising as it removes the external teacher required by on-policy distillation (OPD), yet it still needs asymmetric information between teacher and student to ensure that the self-teacher provides a stronger learning signal than the student.

By Yijun Liang, Yunjie Tian, Yijiang Li, Yuqi Jia, Furong Huang, Tianyi Zhou, Di Fu
efficiencybenchmarks
More like this →
arXiv AI
Sep 16

OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation

OPD‑Aha is a privileged on‑policy distillation method that improves multimodal reasoning by reconstructing the distillation target from the teacher’s isolated visual preference instead of relying on fragile teacher‑student discrepancies. It suppresses continuations that contradict the image, encouraging students to interrupt flawed reasoning with reflection tokens such as "wait" and "actually." This approach leads to consistent improvements across fine‑grained perception and complex multimodal reasoning benchmarks.

By Chenhao Qiu, Dawei Li, Yechao Zhang, Lei Gong, Zhen Tan
efficiencymultimodalbenchmarkssafety
More like this →