arXiv AI
Jun 12
Hellinger Multimodal Variational Autoencoders
arXiv:2601. 06572v4 Announce Type: replace-cross Abstract: Multimodal variational autoencoders (VAEs) are widely used for weakly supervised generative learning with multiple modalities.
Expressivity In Multimodal Contrastive Learning
arXiv:2608. 17203v1 Announce Type: cross Abstract: Contrastive learning has become a cornerstone of modern representation learning, powering CLIP-style models that underpin text-to-image generation, vision-language models, and retrieval across a rapidly growing range of modalities.
Related stories
arXiv Machine Learning
Jul 16
The Hyperspherical Geometry of CLIP Latent Space: A Semantic Mixture Model
arXiv:2607. 13660v1 Announce Type: new Abstract: Contrastive Language-Image Pretraining (CLIP) representations form a semantic embedding space governed by cosine similarity, reflecting an intrinsic hyperspherical geometry.
arXiv AI
Jun 12
Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality
arXiv:2606. 13288v1 Announce Type: cross Abstract: Contrastively trained vision-language models like CLIP, have made remarkable progress in learning joint image-text representations, but still face challenges in compositional understanding.
arXiv Machine Learning
Jun 16
MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation
arXiv:2606. 16408v1 Announce Type: new Abstract: We introduce MUNI, an end-to-end multimodal latent diffusion framework for any-to-any generation that unifies subset-conditioned cross-modal generation and unconditional joint sampling through a shared stochastic latent.
arXiv AI
Jun 8
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
arXiv:2602. 07026v3 Announce Type: replace-cross Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of distinct modalities expressing identical semantics occupy systematically offset regions.
arXiv AI
Jun 30
Multimodal Representation Alignment for Cross-modal Information Retrieval
arXiv:2506. 08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways.
arXiv AI
Jul 7
Transferability Between Understanding and Generation in Unified Multimodal Models
arXiv:2607. 04423v1 Announce Type: cross Abstract: Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains understudied.
arXiv AI
Aug 11
LF${}^{2}$AR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models
arXiv:2503. 06211v3 Announce Type: replace-cross Abstract: Text-pretrained language models (LMs) encode rich world knowledge, but adapting them to process and generate perceptual modalities such as audio and images while effectively leveraging that knowledge remains challenging.
arXiv AI
Jul 21
Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learning
arXiv:2607. 17673v1 Announce Type: cross Abstract: Contrastive learning is increasingly moving toward settings with three or more modalities instead of image-text pairs.
Hugging Face Trending Papers
Jul 5
Transferability Between Understanding and Generation in Unified Multimodal Models
Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains understudied. We investigate $\boldsymbol{\mathsf{transferability}}$ in UMMs: whether training a capability on one task improves the same capability on the other without explicit supervision.
arXiv AI
Jul 28
Controlling Embedding Spaces with Text-Conditioned Transformations
arXiv:2607. 22919v1 Announce Type: cross Abstract: Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross-modal zero-shot classification.
arXiv Machine Learning
Jun 30
Contrastive vision-language learning with paraphrasing and negation
arXiv:2511. 16527v2 Announce Type: replace-cross Abstract: Contrastive vision-language models continue to be the dominant approach for image-text retrieval.