arXiv:2608. 04013v1 Announce Type: cross Abstract: Recent advances in Multimodal Emotion Recognition in Conversations (MERC) highlight its reliance on complete multimodal inputs.
MissMAC-Bench is a new benchmark for evaluating how multimodal affective computing systems handle missing modality data. It establishes fair, unified evaluation standards based on cross‑modal synergy, requiring models to perform without prior missing data during training and to handle both complete and incomplete inputs. The benchmark includes protocols for fixed and random missing patterns at dataset and instance levels, and experiments on three language models across four datasets demonstrate its effectiveness.
arXiv:2609.06188v1 Announce Type: new Abstract: Multimodal sentiment analysis and emotion recognition in conversations demand effective modeling of heterogeneous interactions across textual, acoustic...
arXiv:2607. 00293v1 Announce Type: cross Abstract: Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge.
arXiv:2610. 29430v1 Announce Type: new Abstract: Multimodal multi-label emotion recognition is important for affect-aware systems, but models trained with complete text, visual, and acoustic streams often degrade sharply when some streams are unavailable at deployment.
InfMasking introduces a unified multimodal learning framework that uses Infinite Masking to encourage fusion modules to capture synergistic information across modalities. By randomly masking modality features before fusion and enforcing consistency with an unmasked reference, the method discourages reliance on dominant or shared signals and promotes meaningful cross‑modal interactions. The approach models masked fused representations as Gaussian, optimizing closed‑form variational bounds, and demonstrates improved predictive performance and higher synergy metrics on benchmark datasets.
Recent advances in multimodal large language models (MLLMs) have significantly improved the performance of multimodal emotion recognition (MER) and enabled interpretable description generation by jointly modeling video, audio, and language, etc. However, these performance improvements are often accompanied by an increase in model parameter size (e.
arXiv:2608. 09986v1 Announce Type: new Abstract: Most existing multimodal sentiment analysis approaches assume access to complete multimodal inputs.
arXiv:2606. 05008v1 Announce Type: cross Abstract: As multi-modal models advance towards long-form video understanding, memory emerges as a critical capability.
The paper introduces TTSD‑FAR, a test‑time self‑distillation framework that adapts large video‑language models to missing‑modality scenarios in emotion recognition. A frozen teacher trained on complete modalities guides a low‑rank student, while Fisher‑Anchored Restoration monitors Fisher information to prevent drift and restore the student when distribution shifts occur. Experiments on MELD, DFEW, and BAH with up to 50% missing modalities show TTSD‑FAR consistently outperforms entropy‑based adaptation, retrieval‑augmented generation, and perplexity‑based generation, maintaining performance over long adaptation horizons.
arXiv:2607. 08839v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are typically designed under the assumption that all modalities available during training will also be accessible at inference.
arXiv:2606. 09853v1 Announce Type: new Abstract: A central objective in multimodal learning is to capture synergy: task-relevant information that arises only from the joint use of multiple modalities, and is not available from any single modality alone.