arXiv:2606. 10504v1 Announce Type: new Abstract: Cross-modal knowledge distillation (CMKD) studies how a (large) teacher model trained on one type of data (e.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
Cross-modal knowledge distillation (CMKD) studies how a (large) teacher model trained on one type of data (e. g.
arXiv:2412. 01282v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks.
arXiv:2604. 22823v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) rely on multimodal pre-training over diverse data sources, where different datasets often induce complementary cross-modal alignment capabilities.
LLaVAFlow is an information‑theoretic distillation framework designed to preserve cross‑modal alignment in Multimodal Large Language Models during visual instruction tuning. It compresses the mutual information between extracted relations and MLLM embeddings to refine alignment flow, and then maximizes mutual information between pretrained and fine‑tuned alignment flows to transfer compact alignment information. Experiments demonstrate that LLaVAFlow effectively maintains alignment flow, improving downstream performance and generalization.
arXiv:2609.37243v1 Announce Type: cross Abstract: Cross-modal knowledge distillation transfers knowledge from a teacher modality to a student modality. Existing feature-level alignment methods typica...
CLIP-RD introduces a relational distillation framework for efficient CLIP knowledge distillation, featuring Vertical Relational Distillation (VRD) and Cross Relational Distillation (XRD). VRD aligns intra‑modal similarity distributions between teacher and student, while XRD aligns cross‑modal similarity distributions to enforce bidirectional symmetry. This joint modeling of multidirectional relational structures improves the student’s embedding geometry, yielding a 1.8%p performance gain over CLIP‑KD across various architectures, tasks, and corruption settings with minimal training‑time overhead.