← Back to all news
arXiv Machine Learning By Jiashuo Zou, Xiaobo Xia

Learning What to Trust in Multimodal Learning under Noisy Supervision

Read the original on arXiv Machine Learning →

The paper introduces REFINE, a multimodal label‑noise detection framework that leverages both fused and unimodal representations to identify trustworthy examples in noisy supervision settings. By constructing discriminative eigenvectors for target and background classes, REFINE selects trusted representation spaces and measures alignment of instance representations within these spaces, combining the resulting subsets to provide cleaner supervision for the multimodal classifier. Experiments across various tasks show that REFINE outperforms baseline methods, improving model generalization while reducing the impact of mislabeled data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 22

Generalized Multimodal Foundation Model

The paper introduces a generalized multimodal foundation model that can handle arbitrary combinations of modalities and prediction tasks. It trains on large-scale synthetic multimodal datasets with diverse causal structures to learn transferable multimodal correlations. Experiments on 18 real-world datasets across 12 modalities and 11 tasks show competitive performance compared to specialized models without task-specific adaptation.

By Huizi Cui, Zongbo Han, Chenggong Ding, Naichuan Xiao, Jialong Yang, Jingdong Chen, Guangyu Wang, Qinghua Hu, Changqing Zhang
multimodal
More like this →
arXiv Machine Learning
Sep 24

Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification

The paper identifies that in multimodal learning, optimization often produces asymmetric certainty gains, with the stronger modality becoming more confident than the weaker one, which leads to imbalanced contributions and suboptimal performance. The authors attribute this issue to unimodal characteristics and propose a Max Confidence Regularization (MaxCR) method that tracks each modality’s semantic confidence via a nonlinear sparsity measure and applies max suppression and excitation to balance confidence levels. Experiments on standard datasets demonstrate that MaxCR improves overall performance compared to state‑of‑the‑art multimodal baselines.

By Longfei Huang, Xiangyu Wu, Yang Yang
efficiencymultimodalbenchmarks
More like this →