arXiv Machine Learning
ConsMSA: Semantic Distribution Consistency Learning for Multimodal Sentiment Analysis
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Sep 25
SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Data
SemMSA introduces a latent semantic‑aided framework for multimodal sentiment analysis that leverages large language models to generate rich sentiment‑relevant semantics. The method employs Cross‑modal Semantic Refinement (CSR) to fuse visual, acoustic, and language features in a frozen LLM embedding space, and Cross‑modal Spectral Alignment (CSA) to align these refined semantics with all modalities via spectral enhancement of kernel Gram matrices. Experiments on SIMS, MOSI, and MOSEI benchmarks show that SemMSA achieves state‑of‑the‑art performance.
arXiv AI
Aug 21
Contrastive Mixed Prompt Learning for Incomplete Multimodal Sentiment Analysis with Unseen Modality Combination
arXiv:2608. 20019v1 Announce Type: new Abstract: Incomplete multimodal sentiment analysis has garnered significant attention in recent years.
arXiv Machine Learning
Sep 29
Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis
Reliability-aware Cross-sample Enhancement (RCE) is a framework for multimodal sentiment analysis that tackles noise and missing modalities by first applying an adaptive variational information bottleneck to compress unreliable modality information. It then retrieves high‑confidence, semantically consistent neighbors from a large candidate pool to enrich current representations, and finally fuses cross‑modal interactions through a multilevel reliability‑aware mechanism. Experiments show RCE consistently outperforms state‑of‑the‑art methods in full, noisy, and missing‑modality scenarios.
arXiv AI
Aug 5
Rethinking Modality Reliability in Multimodal Sentiment Analysis with Incomplete Observations
arXiv:2608. 03611v1 Announce Type: new Abstract: Multimodal Sentiment Analysis (MSA) integrates text, audio, and vision to infer human affect, yet real-world multimodal observations are often incomplete.
arXiv Computer Vision
Oct 21
Text-Anchored Modality Experts with Uncertainty-Guided Routing for Multimodal Sentiment Analysis
The paper introduces TAME, a Text‑Anchored Modality Experts framework that treats language as a fixed, always‑active expert and routes audio‑textual and video‑textual cross‑modal branches as dynamic auxiliary experts. It incorporates lightweight Prophet Heads to estimate sample‑level uncertainty for each modality, guiding a reliability‑aware routing mechanism for expert aggregation. Experiments on CMU‑MOSI, CMU‑MOSEI, and CH‑SIMS show that TAME achieves state‑of‑the‑art performance while reducing parameters by up to 78.7% and FLOPs by up to 41.2% compared to a mixture‑of‑experts baseline.
arXiv AI
Aug 12
MIDAS: Mutual Information Disentanglement with Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysis
arXiv:2608. 09986v1 Announce Type: new Abstract: Most existing multimodal sentiment analysis approaches assume access to complete multimodal inputs.