arXiv AI
Dependency, Compression, and Synergy: A Unified Information-Theoretic View of Multimodal Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
2d ago
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
The survey titled "Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey" reviews how Mixture-of-Experts (MoE) frameworks address multimodal learning problems. It categorizes MoE’s contributions into three roles: an efficient multimodal engine that scales without excessive computation, a representation learner that fuses diverse expert knowledge, and an adapter that handles imperfect data such as missing modalities. The paper also highlights research gaps like interpretable routing, expert communication, and lifelong multimodal learning.
arXiv AI
Jun 11
Information-Theoretic Decomposition for Multimodal Interaction Learning
arXiv:2606. 11614v1 Announce Type: cross Abstract: Multimodal learning hinges on capturing redundant, unique, and synergistic information across modalities, which collectively constitute multimodal interactions.
arXiv Machine Learning
Jun 16
SPICE: Synergy and Partial Information Based Curriculum Evolution
arXiv:2606. 16639v1 Announce Type: new Abstract: Multimodal learning exploits complementary information across heterogeneous modalities.
arXiv AI
Oct 9
HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling
The paper introduces HRIL, a method for learning multimodal representations that explicitly models higher‑order interactions among modalities using a cross‑moment tensor and Tucker decomposition. It adds a synergy‑aware regularizer to preserve higher‑order coupling capacity, aiming to capture synergistic signals that only emerge from joint configurations of multiple modalities. Experiments on controlled synergy tasks and real‑world benchmarks show consistent improvements over existing multimodal contrastive methods, especially on tasks dominated by synergistic interactions.
arXiv AI
Jun 2
Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition
arXiv:2606. 00959v1 Announce Type: new Abstract: Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment.
arXiv Machine Learning
Aug 5
DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passing
arXiv:2608. 02769v1 Announce Type: cross Abstract: Multimodal supervised learning seeks to leverage multiple heterogeneous data sources to improve predictive performance.