arXiv AI June 29, 2026 By Thomas Shih-Chao Liang, Zhuoran Yu, Yong Jae Lee

Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge

arXiv:2606. 27527v1 Announce Type: cross Abstract: Large Language Models (LLMs) possess broad conceptual knowledge acquired through large-scale text pretraining, yet their potential to supervise models in other modalities remains underexplored.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Aug 24

When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQA

arXiv:2511.17886v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) have achieved remarkable success across multimodal tasks, yet their substantial computational demands hinder ef...

By Pume Tuchinda, Parinthapat Pengpun, Romrawin Chumpu, Patomporn Payoungkhamdee, Sarana Nutanong, Peerat Limkonchotiwat

llms nlp efficiency multimodal benchmarks

arXiv AI

4d ago

Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes

arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a froze...

By Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris

llms efficiency multimodal benchmarks

arXiv AI

Jun 3

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

arXiv:2605. 18740v4 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) still struggle with fine-grained visual understanding, where answers often depend on small but decisive evidence in the full image.

By Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, Yaojie Lu

llms agents efficiency multimodal benchmarks

arXiv AI

Jun 2

Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning

arXiv:2606. 00105v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress on vision-language tasks, but they may also memorize and expose sensitive or restricted knowledge, raising concerns about privacy and broader safety risks.

By Junkai Chen, Yuhao He, Junxiang You, Ruiqi Liu, Chenyu Wang, Shu Wu

llms efficiency multimodal safety

arXiv Machine Learning

Aug 4

Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models

arXiv:2608. 01263v1 Announce Type: new Abstract: On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions at prefixes along those trajectories.

By Leyan Xue, Feng Xiong, Mingjun Ma, Changqing Zhang

llms efficiency multimodal benchmarks

arXiv Computation and Language

5d ago

MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models

arXiv:2609.39920v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) exhibit strong multimodal in-context learning (ICL) capabilities, yet this ability degrades substantially as mod...

By Yanshu Li, Jiaqian Li, Canran Xiao, Xi Xiao, Tianyang Wang, Yongtai Liu

llms efficiency multimodal benchmarks safety

One email a morning, machine-written

Related stories

When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQA

Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning

Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models

MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models