arXiv Computer Vision
Rethinking Gradient Modulation in Multimodal Regression
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Machine Learning
Sep 24
Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification
The paper identifies that in multimodal learning, optimization often produces asymmetric certainty gains, with the stronger modality becoming more confident than the weaker one, which leads to imbalanced contributions and suboptimal performance. The authors attribute this issue to unimodal characteristics and propose a Max Confidence Regularization (MaxCR) method that tracks each modality’s semantic confidence via a nonlinear sparsity measure and applies max suppression and excitation to balance confidence levels. Experiments on standard datasets demonstrate that MaxCR improves overall performance compared to state‑of‑the‑art multimodal baselines.
arXiv AI
Jul 10
MultiFair: Multimodal Balanced Fairness-Aware Medical Classification with Dual-Level Gradient Modulation
arXiv:2510. 07328v2 Announce Type: replace-cross Abstract: Medical decision systems increasingly rely on data from multiple sources to ensure reliable and unbiased diagnosis.
arXiv Machine Learning
Jul 7
Beyond Modality Fusion: Deep Ensembles for Multimodal Classification
arXiv:2607. 05019v1 Announce Type: new Abstract: In multimodal classification, late-fusion approaches classify concatenated modality-specific features extracted by unimodal neural networks.
arXiv Machine Learning
6d ago
Fusion or Confusion? Multimodal Complexity Is Not All You Need
The paper reexamines the belief that more complex multimodal neural architectures lead to better performance. By reimplementing 19 influential multimodal methods across nine datasets with up to 23 modalities, the authors find that increased architectural complexity often causes confusion rather than effective fusion, and such models rarely surpass unimodal baselines or a simple baseline (SimBaMM). They also highlight methodological flaws in top-tier multimodal studies and call for a shift toward standardized evaluation and methodological rigor.
arXiv Machine Learning
Sep 22
Generalized Multimodal Foundation Model
The paper introduces a generalized multimodal foundation model that can handle arbitrary combinations of modalities and prediction tasks. It trains on large-scale synthetic multimodal datasets with diverse causal structures to learn transferable multimodal correlations. Experiments on 18 real-world datasets across 12 modalities and 11 tasks show competitive performance compared to specialized models without task-specific adaptation.
arXiv Machine Learning
Sep 24
VCMM: Variance-Calibrated Momentum for Multimodal Learning
VCMM: Variance-Calibrated Momentum for Multimodal Learning proposes a new optimizer that adapts momentum based on modality-specific gradient dynamics. It estimates minibatch noise and temporal drift online, using a Kalman-inspired controller to set modality-specific momentum and applies bias correction for the first moment. Experiments on four multimodal benchmarks show consistent improvements with modest training overhead.