SemMSA introduces a latent semantic‑aided framework for multimodal sentiment analysis that leverages large language models to generate rich sentiment‑relevant semantics. The method employs Cross‑modal Semantic Refinement (CSR) to fuse visual, acoustic, and language features in a frozen LLM embedding space, and Cross‑modal Spectral Alignment (CSA) to align these refined semantics with all modalities via spectral enhancement of kernel Gram matrices. Experiments on SIMS, MOSI, and MOSEI benchmarks show that SemMSA achieves state‑of‑the‑art performance.
By Wenhao Li, Zhibin Wu, Chong Xiao, Qiangchang Wang
arXiv:2608. 20019v1 Announce Type: new Abstract: Incomplete multimodal sentiment analysis has garnered significant attention in recent years.
By Kaixin Xu, NaiJin Liu, Yulin Kang, Tangyue Jin, Zixuan Yu, Wenxi Zhao, Yibei Liu, Qianle Zhang, Yangyang Wu, Mengying Zhu, Meng Xi
Reliability-aware Cross-sample Enhancement (RCE) is a framework for multimodal sentiment analysis that tackles noise and missing modalities by first applying an adaptive variational information bottleneck to compress unreliable modality information. It then retrieves high‑confidence, semantically consistent neighbors from a large candidate pool to enrich current representations, and finally fuses cross‑modal interactions through a multilevel reliability‑aware mechanism. Experiments show RCE consistently outperforms state‑of‑the‑art methods in full, noisy, and missing‑modality scenarios.
By Menghua Jiang, Haokai Gao, Xiangui Kang, Haifeng Hu, Sijie Mai
arXiv:2608. 03611v1 Announce Type: new Abstract: Multimodal Sentiment Analysis (MSA) integrates text, audio, and vision to infer human affect, yet real-world multimodal observations are often incomplete.
By Chunlei Meng, Jacqueline J. Pang, Pengbin Feng, Zhenyu Yu, Chun Ouyang, Zhongxue Gan
The paper introduces TAME, a Text‑Anchored Modality Experts framework that treats language as a fixed, always‑active expert and routes audio‑textual and video‑textual cross‑modal branches as dynamic auxiliary experts. It incorporates lightweight Prophet Heads to estimate sample‑level uncertainty for each modality, guiding a reliability‑aware routing mechanism for expert aggregation. Experiments on CMU‑MOSI, CMU‑MOSEI, and CH‑SIMS show that TAME achieves state‑of‑the‑art performance while reducing parameters by up to 78.7% and FLOPs by up to 41.2% compared to a mixture‑of‑experts baseline.
By Xiaohua Huang, Bingrui Lei, Xiuyan hao, Ling Zhou, Yuan Zong, Wenming Zheng, Guoying Zhao
arXiv:2608. 09986v1 Announce Type: new Abstract: Most existing multimodal sentiment analysis approaches assume access to complete multimodal inputs.
By Yuhua Wen, Yingying Zhou, Qifei Li, Yingming Gao, Zhengqi Wen, Jianhua Tao, Ya Li
arXiv:2608. 19971v1 Announce Type: new Abstract: Multimodal sentiment analysis aims to infer affective states by integrating language, visual, and acoustic cues.
By Zhifa Geng, Subin Huang, Hao Guo, Junjie Chen, Sanmin Liu, Chao Kong
The paper presents a method for robust multimodal sentiment analysis that handles incomplete or noisy modalities. It introduces a completeness estimation technique to measure how much sentiment-relevant information remains in partial data, guiding the reconstruction of missing semantics. A joint training strategy stabilizes multi-task learning for sentiment prediction and completeness estimation, and experiments on three benchmark datasets show improved semantic reconstruction and sentiment accuracy.
By Han-Jun Choi, Byunggill Joe, Saim Shin, Jin Yea Jang
I$^2$-KAN is a new multimodal sentiment analysis framework that separates intra‑modal and pairwise inter‑modal information into distinct pathways. Each pathway is refined by a Task‑Decoupled Linear‑Gated Denoising Bottleneck and fused using a structured Kolmogorov‑Arnold Network, enabling explicit pathway associations and multi‑level interpretation. Experiments on CMU‑MOSI, CMU‑MOSEI, and CH‑SIMS v2 show strong performance and reveal that a modality’s standalone predictive power does not always reflect its interaction‑conditioned utility.
By Xiaoting Xie, Feng Qiao, Xianwei Zhang, Hailong Chen, Kaiwen Zheng, Joemon M. Jose, Xuri Ge, Yujun Li
The paper introduces Uncertainty‑Guided Multimodal Consensus (UMC) for multimodal sentiment analysis. UMC first recovers continuous unimodal pseudo‑labels using Continuous Prototype‑Guided Label Recovery (CPLR), which blends heteroscedastic predictions with ordered sentiment prototypes and refines them with confidence and label conflict. It then forms a reliability‑aware consensus across modalities via Reliability and Precision Consensus (RPC), which estimates sample‑dependent modality reliability from predictive statistics and cross‑modal disagreement and combines it with predictive precision through a closed‑form solver. Experiments on four public MSA datasets show that UMC outperforms existing methods.
By Guowei Zhong, Junjie Li, Huaiyu Zhu, Ruohong Huan, Yun Pan
arXiv:2608. 16201v1 Announce Type: new Abstract: Multimodal sentiment analysis (MSA) aims to predict sentiment polarity and intensity from heterogeneous inputs such as text, audio, and vision.
By Shanshan Lin, Yuesheng Wu, Chao Chen, Yizhe Yang, Zhihao Chen, Zexian Yang, Xiangwen Liao
arXiv:2610.20543v1 Announce Type: new
Abstract: Multimodal sentiment models are usually evaluated on complete utterances, yet streams often lose a contiguous span of text, audio, or video. We study l...
By Minghao Kong, Deng Yao, Xi Zhang, Ying Gao, Jiurun Chen, Zirong Liu, Rongjie Wang