← Back to all news
arXiv Machine Learning By Mary Ogbuka Kenneth, Foaad Khosmood, Abbas Edalat

Mitigating Strong-Modality Collapse in Multimodal Learning via Inverted Asymmetric Fusion

Read the original on arXiv Machine Learning →

The paper introduces Inverted Asymmetric Fusion (IAF) to address strong-modality collapse in multimodal learning, where dominant modalities are degraded during fusion. IAF preserves the dominant modality by passing it unchanged and letting weaker modalities attend to it, while also strengthening weaker modalities via Modality-Aware Knowledge Distillation. Experiments on MultiHuSE, UR-FUNNY, and MUStARD show that IAF maintains unimodal performance and improves over the best unimodal baseline by up to 8.25%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 24

Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification

The paper identifies that in multimodal learning, optimization often produces asymmetric certainty gains, with the stronger modality becoming more confident than the weaker one, which leads to imbalanced contributions and suboptimal performance. The authors attribute this issue to unimodal characteristics and propose a Max Confidence Regularization (MaxCR) method that tracks each modality’s semantic confidence via a nonlinear sparsity measure and applies max suppression and excitation to balance confidence levels. Experiments on standard datasets demonstrate that MaxCR improves overall performance compared to state‑of‑the‑art multimodal baselines.

By Longfei Huang, Xiangyu Wu, Yang Yang
efficiencymultimodalbenchmarks
More like this →
arXiv Computation and Language
Oct 1

Fusion Anything: A Generalized Multimodal Foundation Model

The paper introduces Fusion Anything Model (FAM), a foundation model designed for generalized multimodal data fusion that can handle arbitrary modality combinations and prediction tasks. FAM is trained on large-scale synthetic multimodal datasets generated via Structural Multimodal Causal Models (SMCMs), enabling it to encode transferable multimodal correlations. Experiments on 18 real-world datasets across 12 modalities and 11 tasks show that FAM performs competitively with specialized models without requiring task-specific adaptation.

By Huizi Cui, Zongbo Han, Chenggong Ding, Naichuan Xiao, Jialong Yang, Jingdong Chen, Guangyu Wang, Qinghua Hu, Changqing Zhang
multimodal
More like this →