The paper introduces the Mult2EMo dataset, which gathers annotations from both authors and readers on multimodal social media posts and the real‑world events that triggered them. It investigates how well readers can reconstruct the authors’ emotional experience from the post content, emphasizing the importance of both text and image modalities. The study finds that accurate emotion reconstruction is possible but remains challenging, especially when images dominate the expression and when understanding the triggering event is essential.
arXiv:2609.24215v1 Announce Type: new Abstract: Although text-to-image models can accurately depict subjects and scenes, creators still struggle to specify the fine-grained emotions an image should c...
arXiv:2505.24427v2 Announce Type: replace Abstract: Accurate modeling of subjective phenomena such as emotion expression requires data annotated with authors' intentions. Commonly such data is collec...
arXiv:2606. 05816v1 Announce Type: cross Abstract: T2I models cannot effectively capture sentiment from various types of text, including diaries, as they primarily focus on visual object-related patterns rather than contextual emotional understanding.
The paper introduces CLEAR, a framework for Multimodal Emotion Recognition in Conversation that couples contextual representations with local discriminative cues. It replaces emotion label generation with continuous internal representations from a vision‑language model and trains a classifier on these representations. Sample‑adaptive residual coupling is used to integrate textual and acoustic evidence from the target utterance into the contextual representation, improving emotion recognition performance on IEMOCAP and MELD datasets.
arXiv:2609.26907v1 Announce Type: cross Abstract: Memes often derive their harmful, hateful, or sarcastic meaning from small but decisive visual, textual, or cross-modal cues. Existing multimodal cla...
arXiv:2602. 16161v4 Announce Type: replace-cross Abstract: Emotional expression underpins natural communication and effective human-computer interaction.
Cross-Modal Emotion Understanding: A Transformer-GAT Approach for Dialogue Emotion Recognition proposes a hybrid framework that combines a Transformer and a Graph Attention Network to capture both global semantic information and fine-grained relationships between modalities. The model is evaluated on the IEMOCAP and MELD datasets, achieving weighted F1 scores of 72.45% and 77.37%, respectively, and surpasses state‑of‑the‑art methods. These results suggest that integrating multimodal features with balanced global and local context modeling can provide deeper emotional insights for dialogue emotion recognition.
arXiv:2607. 14769v1 Announce Type: cross Abstract: Existing text summarization research has focused much on monologic information (e.
arXiv:2609.06188v1 Announce Type: new Abstract: Multimodal sentiment analysis and emotion recognition in conversations demand effective modeling of heterogeneous interactions across textual, acoustic...
arXiv:2609.36563v1 Announce Type: new Abstract: Visual emotion recognition commonly assumes that all evidence required for prediction is contained in the observed image or video. Yet the same visible...
The paper compares text‑based and feature‑based models for recognizing compound emotions in real‑world videos. It proposes textualizing non‑verbal cues from audio and visual modalities into text to leverage large language models, while feature‑based models directly combine extracted multimodal features. Experiments on the C‑EXPR‑DB dataset show that feature‑based models outperform textualization in the wild, though textual models can excel when rich transcripts are available.
The paper introduces TIC‑Bench, a new benchmark for evaluating multimodal large language models on deeply interleaved text‑image contexts. It covers logical, temporal, and spatial association tasks, totaling 2,280 questions across eight specific types. The authors benchmarked ten state‑of‑the‑art MLLMs, finding a significant performance gap versus human experts and highlighting persistent challenges in integrating evidence across interleaved visual and textual inputs.