Hugging Face Trending Papers
Jun 9
Cross-Modal Knowledge Distillation without Paired Data: Theoretical Foundation and Algorithm
Cross-modal knowledge distillation (CMKD) studies how a (large) teacher model trained on one type of data (e. g.
Cross-Modal Knowledge Distillation without Paired Data: Theoretical Foundation and Algorithm
arXiv:2606. 10504v1 Announce Type: new Abstract: Cross-modal knowledge distillation (CMKD) studies how a (large) teacher model trained on one type of data (e.
Related stories
arXiv AI
Jun 3
Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement
arXiv:2412. 01282v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks.
arXiv AI
Jul 14
PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging
arXiv:2604. 22823v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) rely on multimodal pre-training over diverse data sources, where different datasets often induce complementary cross-modal alignment capabilities.
arXiv AI
Jun 8
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
arXiv:2602. 07026v3 Announce Type: replace-cross Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of distinct modalities expressing identical semantics occupy systematically offset regions.
arXiv Machine Learning
Jul 2
Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learning
arXiv:2505. 19614v2 Announce Type: replace Abstract: Multimodal learning has seen remarkable progress, particularly with large-scale pre-training across various modalities.
arXiv AI
Jul 7
SpectralGCD: Spectral Concept Selection and Cross-modal Representation Learning for Generalized Category Discovery
arXiv:2602. 17395v2 Announce Type: replace-cross Abstract: Generalized Category Discovery (GCD) aims to identify novel categories in unlabeled data while leveraging a small labeled subset of known classes.
Hugging Face Trending Papers
Jul 23
UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging
Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions.
arXiv AI
Jun 29
Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge
arXiv:2606. 27527v1 Announce Type: cross Abstract: Large Language Models (LLMs) possess broad conceptual knowledge acquired through large-scale text pretraining, yet their potential to supervise models in other modalities remains underexplored.
arXiv AI
Jun 30
Multimodal Representation Alignment for Cross-modal Information Retrieval
arXiv:2506. 08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways.
Hugging Face Trending Papers
Jun 9
When to Align, When to Predict: A Phase Diagram for Multimodal Learning
Cross-modal alignment (CA) and cross-modal prediction (CP) are the dominant paradigms for multimodal representation learning, yet there is no systematic understanding of when each succeeds, when each fails, and when cross-modal training helps at all -- a gap that leaves practitioners, especially in scientific domains like biomedicine or astrophysics, with heterogeneous instruments and multiple levels of organization and measurement, unable to diagnose why standard methods underperform the best single modality. We develop a unified linear framework that addresses both questions.
arXiv Machine Learning
Jun 10
When to Align, When to Predict: A Phase Diagram for Multimodal Learning
arXiv:2606. 11190v1 Announce Type: new Abstract: Cross-modal alignment (CA) and cross-modal prediction (CP) are the dominant paradigms for multimodal representation learning, yet there is no systematic understanding of when each succeeds, when each fails, and when cross-modal training helps at all -- a gap that leaves practitioners, especially in scientific domains like biomedicine or astrophysics, with heterogeneous instruments and multiple levels of organization and measurement, unable to diagnose why standard methods underperform the best single modality.
arXiv Machine Learning
Jul 13
Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing
arXiv:2607. 08839v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are typically designed under the assumption that all modalities available during training will also be accessible at inference.