arXiv:2608. 03611v1 Announce Type: new Abstract: Multimodal Sentiment Analysis (MSA) integrates text, audio, and vision to infer human affect, yet real-world multimodal observations are often incomplete.
By Chunlei Meng, Jacqueline J. Pang, Pengbin Feng, Zhenyu Yu, Chun Ouyang, Zhongxue Gan
Reliability-aware Cross-sample Enhancement (RCE) is a framework for multimodal sentiment analysis that tackles noise and missing modalities by first applying an adaptive variational information bottleneck to compress unreliable modality information. It then retrieves high‑confidence, semantically consistent neighbors from a large candidate pool to enrich current representations, and finally fuses cross‑modal interactions through a multilevel reliability‑aware mechanism. Experiments show RCE consistently outperforms state‑of‑the‑art methods in full, noisy, and missing‑modality scenarios.
By Menghua Jiang, Haokai Gao, Xiangui Kang, Haifeng Hu, Sijie Mai
arXiv:2606. 02679v1 Announce Type: new Abstract: Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed.
By Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang, Xinpei Wang, Weitong Chen
RiVaT‑Fuse introduces a reliability‑calibrated variational tensor fusion framework for multimodal image‑metadata prediction, treating fusion as a sample‑wise latent‑state estimation rather than simple aggregation. It replaces scalar modality confidence with matrix‑valued trust geometry, decomposes interactions into additive, multiplicative, and relational components, and couples the latent state with conditional robustness and structured multi‑task prediction. On an image‑level benchmark, RiVaT‑Fuse outperforms direct representation‑level baselines and improves probability and label stability under perturbation.
By Yingfan Xu, Tieming Liu, Ye Liang, Taiping Liu
The paper identifies that in multimodal learning, optimization often produces asymmetric certainty gains, with the stronger modality becoming more confident than the weaker one, which leads to imbalanced contributions and suboptimal performance. The authors attribute this issue to unimodal characteristics and propose a Max Confidence Regularization (MaxCR) method that tracks each modality’s semantic confidence via a nonlinear sparsity measure and applies max suppression and excitation to balance confidence levels. Experiments on standard datasets demonstrate that MaxCR improves overall performance compared to state‑of‑the‑art multimodal baselines.
By Longfei Huang, Xiangyu Wu, Yang Yang
arXiv:2606. 00959v1 Announce Type: new Abstract: Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment.
By Wanlong Fang, Tianle Zhang, Wen Tao, Alvin Chan