arXiv AI
LLaVA-CKD: Bottom-Up Cascaded Knowledge Distillation for Vision-Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 3
Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement
arXiv:2412. 01282v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks.
arXiv Computation and Language
Aug 24
When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQA
arXiv:2511.17886v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) have achieved remarkable success across multimodal tasks, yet their substantial computational demands hinder ef...
arXiv AI
Jun 29
Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge
arXiv:2606. 27527v1 Announce Type: cross Abstract: Large Language Models (LLMs) possess broad conceptual knowledge acquired through large-scale text pretraining, yet their potential to supervise models in other modalities remains underexplored.
arXiv Machine Learning
Aug 4
Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models
arXiv:2608. 01263v1 Announce Type: new Abstract: On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions at prefixes along those trajectories.
arXiv Machine Learning
Sep 15
Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation
arXiv:2609.13199v1 Announce Type: new Abstract: Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher mo...
arXiv AI
Aug 11
DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation
arXiv:2605. 15532v3 Announce Type: replace-cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics or aggregated from off-the-shelf datasets.