← Back to all news
arXiv Computation and Language By Guowei Zhong, Junjie Li, Huaiyu Zhu, Ruohong Huan, Yun Pan

Uncertainty-Guided Multimodal Consensus for Multimodal Sentiment Analysis

Read the original on arXiv Computation and Language →

The paper introduces Uncertainty‑Guided Multimodal Consensus (UMC) for multimodal sentiment analysis. UMC first recovers continuous unimodal pseudo‑labels using Continuous Prototype‑Guided Label Recovery (CPLR), which blends heteroscedastic predictions with ordered sentiment prototypes and refines them with confidence and label conflict. It then forms a reliability‑aware consensus across modalities via Reliability and Precision Consensus (RPC), which estimates sample‑dependent modality reliability from predictive statistics and cross‑modal disagreement and combines it with predictive precision through a closed‑form solver. Experiments on four public MSA datasets show that UMC outperforms existing methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Sep 29

Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis

Reliability-aware Cross-sample Enhancement (RCE) is a framework for multimodal sentiment analysis that tackles noise and missing modalities by first applying an adaptive variational information bottleneck to compress unreliable modality information. It then retrieves high‑confidence, semantically consistent neighbors from a large candidate pool to enrich current representations, and finally fuses cross‑modal interactions through a multilevel reliability‑aware mechanism. Experiments show RCE consistently outperforms state‑of‑the‑art methods in full, noisy, and missing‑modality scenarios.

By Menghua Jiang, Haokai Gao, Xiangui Kang, Haifeng Hu, Sijie Mai
nlpmultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 21

Text-Anchored Modality Experts with Uncertainty-Guided Routing for Multimodal Sentiment Analysis

The paper introduces TAME, a Text‑Anchored Modality Experts framework that treats language as a fixed, always‑active expert and routes audio‑textual and video‑textual cross‑modal branches as dynamic auxiliary experts. It incorporates lightweight Prophet Heads to estimate sample‑level uncertainty for each modality, guiding a reliability‑aware routing mechanism for expert aggregation. Experiments on CMU‑MOSI, CMU‑MOSEI, and CH‑SIMS show that TAME achieves state‑of‑the‑art performance while reducing parameters by up to 78.7% and FLOPs by up to 41.2% compared to a mixture‑of‑experts baseline.

By Xiaohua Huang, Bingrui Lei, Xiuyan hao, Ling Zhou, Yuan Zong, Wenming Zheng, Guoying Zhao
nlpmultimodalbenchmarks
More like this →