The paper introduces REFINE, a multimodal label‑noise detection framework that leverages both fused and unimodal representations to identify trustworthy examples in noisy supervision settings. By constructing discriminative eigenvectors for target and background classes, REFINE selects trusted representation spaces and measures alignment of instance representations within these spaces, combining the resulting subsets to provide cleaner supervision for the multimodal classifier. Experiments across various tasks show that REFINE outperforms baseline methods, improving model generalization while reducing the impact of mislabeled data.
arXiv:2505. 19614v2 Announce Type: replace Abstract: Multimodal learning has seen remarkable progress, particularly with large-scale pre-training across various modalities.
arXiv:2607. 16789v1 Announce Type: new Abstract: Real-world perception and decision making are inherently multimodal, integrating complementary signals across modalities.
The paper introduces a generalized multimodal foundation model that can handle arbitrary combinations of modalities and prediction tasks. It trains on large-scale synthetic multimodal datasets with diverse causal structures to learn transferable multimodal correlations. Experiments on 18 real-world datasets across 12 modalities and 11 tasks show competitive performance compared to specialized models without task-specific adaptation.
The paper identifies that in multimodal learning, optimization often produces asymmetric certainty gains, with the stronger modality becoming more confident than the weaker one, which leads to imbalanced contributions and suboptimal performance. The authors attribute this issue to unimodal characteristics and propose a Max Confidence Regularization (MaxCR) method that tracks each modality’s semantic confidence via a nonlinear sparsity measure and applies max suppression and excitation to balance confidence levels. Experiments on standard datasets demonstrate that MaxCR improves overall performance compared to state‑of‑the‑art multimodal baselines.
arXiv:2512.20257v2 Announce Type: replace Abstract: With the rise of easily accessible generative tools for creating and manipulating multimedia content, the threat of realistic synthetic alterations...
arXiv:2607. 05019v1 Announce Type: new Abstract: In multimodal classification, late-fusion approaches classify concatenated modality-specific features extracted by unimodal neural networks.
arXiv:2607. 07907v1 Announce Type: cross Abstract: With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associations that originate from their training data.
arXiv:2606. 11190v1 Announce Type: new Abstract: Cross-modal alignment (CA) and cross-modal prediction (CP) are the dominant paradigms for multimodal representation learning, yet there is no systematic understanding of when each succeeds, when each fails, and when cross-modal training helps at all -- a gap that leaves practitioners, especially in scientific domains like biomedicine or astrophysics, with heterogeneous instruments and multiple levels of organization and measurement, unable to diagnose why standard methods underperform the best single modality.
arXiv:2602. 17395v2 Announce Type: replace-cross Abstract: Generalized Category Discovery (GCD) aims to identify novel categories in unlabeled data while leveraging a small labeled subset of known classes.
CRANE is a post‑hoc, model‑agnostic framework that generates multimodal counterfactual explanations by retrieving real training examples from the target class. It uses nearest‑neighbour retrieval across modalities, with Multimodal Consensus Retrieval (MC‑R) intersecting per‑modality neighbour sets to find instances jointly close to the query. Experiments on four heterogeneous datasets and three fusion strategies show MC‑R consistently outperforms other multimodal strategies and remains competitive when a single modality dominates.
arXiv:2606. 18390v1 Announce Type: new Abstract: Motivation: Noisy labels are a common challenge in molecular property prediction because molecular annotations are often obtained from assays, curated databases, or weak annotation pipelines rather than directly observed clean biological states.
arXiv:2610.07928v1 Announce Type: cross Abstract: Dynamic multimodal learning aims to learn robust representations by adaptively modeling information discrepancies across modalities. However, existin...