The paper introduces TRACE, a cognition-oriented framework that models affective episodes as a sequence of Condition, Affect, and Effect stages, incorporating observable cues and cognitive factors like appraisal and regulation. TRACE-Bench evaluates multimodal models on five tasks—grounded affect recognition, regulation decoding, cause reasoning, effect reasoning, and full-chain reconstruction—using 3,746 question-answer pairs from 646 videos. The authors also present TRACER, a structured reasoning method that links each inference to explicit premises, achieving superior performance across all tasks compared to existing baselines.
arXiv:2608. 10448v1 Announce Type: new Abstract: Multimodal emotion recognition in conversation (MERC) requires understanding complex interactions between verbal and non-verbal cues.
arXiv:2609.36563v1 Announce Type: new Abstract: Visual emotion recognition commonly assumes that all evidence required for prediction is contained in the observed image or video. Yet the same visible...
The paper introduces a new paradigm for multimodal emotion understanding that moves beyond surface-level cue-label associations by incorporating cognitive appraisal reasoning. It presents CogEmo-40K, a large instruction‑tuning dataset that elicits evidence‑grounded reasoning across six appraisal dimensions; CogEmo‑MoE, a compact sparse multimodal large language model with interleaved MoE blocks for appraisal‑specific adaptation; and CogEmo‑Bench, a benchmark featuring an Appraisal Evidence Quality Score (AEQS) to evaluate why emotions arise. Experiments demonstrate that this perception‑to‑appraisal approach improves cross‑domain generalization and aligns model reasoning more closely with human cognitive processes.
arXiv:2508.05880v3 Announce Type: replace-cross Abstract: Understanding human emotions is central to user-facing AI applications, safety alignment, and the simulation of human behavior. As emotional...
Affective Image Content Analysis (AICA) aims to recognize and understand emotions elicited by visual content, representing an indispensable step toward Artificial General Intelligence (AGI). However, despite the rapid progress of Multimodal Large Language Models (MLLMs), systematic evaluation of their visual emotional intelligence remains largely absent from recent model releases.
The paper introduces DAN, a training‑free inference‑time framework that improves affective reasoning in multimodal large language models. It combines a Hierarchical Emotional Reasoning Chain (HERC) to better capture fine‑grained visual cues and a Contrastive Discriminative Visual Pruning (CDVP) module to isolate discriminative tokens for semantically similar emotions. Experiments show significant gains, notably a +10.47% improvement on the WebEmo25 benchmark with Qwen3‑VL‑8B‑Instruct.
AffectOmni is a reinforcement‑learning‑trained framework that enhances multimodal large language models for affective reasoning in social and art‑related scenes. It introduces People Focus and Temporal Order rewards to prioritize people‑centric cues and structured reasoning, and uses within‑group comparative scoring for more discriminative rewards. A Thinking Summarizer converts rationales into executable evidence instructions, which are grounded into pixel‑level regions via SAM3, enabling external auditability.
arXiv:2604. 15280v2 Announce Type: replace-cross Abstract: Understanding emotions is a fundamental ability for intelligent systems to be able to interact with humans.
arXiv:2605. 16739v2 Announce Type: replace-cross Abstract: Decoding visual experience from brain activity has advanced substantially, but current brain-to-text systems largely recover semantic content while discarding affect.
HUG‑VIS is a unified multimodal benchmark for human‑centered visual intelligence, comprising 8,400 half‑body videos of 30 professional actors performing 280 emotion‑action prompts in Mandarin. The dataset provides synchronized video, audio, text, and alpha mattes for four tasks—human emotion recognition, video generation, voice cloning, and video matting—allowing evaluation of both open‑ and closed‑source models under a zero‑shot protocol. Results reveal that linguistic cues dominate emotion recognition, visual affect is weakest, and that automatic metrics and human judgments diverge in generation and cloning tasks, while motion‑related boundary fidelity remains a key challenge for matting.
arXiv:2609.36776v1 Announce Type: new Abstract: Emotional Video Captioning aims to generate factually accurate and emotionally empathetic descriptions. While recent methods have recognized the import...
arXiv:2610.18305v1 Announce Type: cross Abstract: The world encompasses both physical environments and human actors. Predicting its evolution requires understanding both physical laws and how emotion...