arXiv Machine Learning
Diverse via bounded Agreement: Geometric Regularization for Multimodal Fusion
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 7
Dynamic Alignment and Calibration for Multimodal Learning
arXiv:2610.07928v1 Announce Type: cross Abstract: Dynamic multimodal learning aims to learn robust representations by adaptively modeling information discrepancies across modalities. However, existin...
arXiv AI
Oct 16
MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data
The paper introduces Modality Self-Play (MSP), a training framework that enables multimodal large language models to learn cross‑modal interactions without requiring paired non‑text data. MSP uses proxy‑mediated textual descriptions of unseen modalities as semantic bridges, allowing a model to generate and align these proxies with observed modalities. Experiments show that MSP achieves state‑of‑the‑art or competitive performance on several multimodal benchmarks and demonstrates effective zero‑shot modality composition through learned bridges and anchor‑supporting proxy descriptions.
arXiv Machine Learning
Sep 22
Generalized Multimodal Foundation Model
The paper introduces a generalized multimodal foundation model that can handle arbitrary combinations of modalities and prediction tasks. It trains on large-scale synthetic multimodal datasets with diverse causal structures to learn transferable multimodal correlations. Experiments on 18 real-world datasets across 12 modalities and 11 tasks show competitive performance compared to specialized models without task-specific adaptation.
arXiv Machine Learning
Jul 2
Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learning
arXiv:2505. 19614v2 Announce Type: replace Abstract: Multimodal learning has seen remarkable progress, particularly with large-scale pre-training across various modalities.
arXiv Machine Learning
Jul 7
Beyond Modality Fusion: Deep Ensembles for Multimodal Classification
arXiv:2607. 05019v1 Announce Type: new Abstract: In multimodal classification, late-fusion approaches classify concatenated modality-specific features extracted by unimodal neural networks.
arXiv AI
Jun 8
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
arXiv:2602. 07026v3 Announce Type: replace-cross Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of distinct modalities expressing identical semantics occupy systematically offset regions.