arXiv Computer Vision

AffectReveal: Event-Grounded Emotion Recognition Beyond Visual Appearances

arXiv Computer Vision
4d ago

Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning

The paper introduces DAN, a training‑free inference‑time framework that improves affective reasoning in multimodal large language models. It combines a Hierarchical Emotional Reasoning Chain (HERC) to better capture fine‑grained visual cues and a Contrastive Discriminative Visual Pruning (CDVP) module to isolate discriminative tokens for semantically similar emotions. Experiments show significant gains, notably a +10.47% improvement on the WebEmo25 benchmark with Qwen3‑VL‑8B‑Instruct.

By Cheng Ye, Weidong Chen, Zhaobo Qi, Beier Zhu, Zhendong Mao
arXiv AI
4d ago

VISTA: Value-Informed Event Appraisal for Multimodal Emotion Conflict

VISTA (Value-Informed Semantic Trust Arbitration) is a learned seven-field appraisal interface that conditions modality arbitration on concerns, event relations, and expression conditions while retaining a joint-evidence residual. It uses a log-odds decomposition to separate emotion expectation from cue diagnosticity, allowing appraisal to change how evidence is interpreted. With a shared Qwen2.5-Omni-7B backbone, VISTA achieves 64.5% conflict accuracy on CA-MER, improving on modality gating by 2.5 percentage points on conflict and 0.2 on consistency, and a frozen-backbone probe reaches 0.600 macro CCC for appraisal readout versus 0.505 for emotion-only fine-tuning.

By Jiale Dai, Liuxian Ma, Xiaoke Niu, Wenjing Zhang, Huiying Zhao, Zhaoxiang Liu, Shiguo Lian, Guojie Song
arXiv Computer Vision
Sep 3

Video2Reaction: Training Foundation Video Models to Predict Audience Reaction

Video2Reaction is a multimodal dataset that links short movie segments to the emotional reactions of viewers, gathered from social media comments. The dataset models reactions as distributions over categorical emotions, capturing the subjective and ambiguous nature of emotional perception. Experiments show that vision‑language models fine‑tuned with LoRA learn effectively from Video2Reaction and outperform specialized baselines, and that models pre‑fine‑tuned on this dataset transfer well to other emotion prediction tasks.

By Sidong Zhang, Trang Nguyen, Shiv Shankar, Gauri Jagatap, Deepak Chandran, Andrea Fanelli, Madalina Fiterau
arXiv Computer Vision
Sep 3

AffectDelta: Beyond Emotion Labels for Image Editing

AffectDelta is a new image editing framework that moves beyond single emotion labels by modeling edits as transitions between eight‑dimensional emotion distributions. It uses a frozen Emotion Distribution Predictor to estimate the source state and a signed difference vector to encode the desired change, which is then translated into context‑dependent semantic and appearance modifications via a transition encoder and a diffusion backbone. The authors introduce AffectPair‑249K, a dataset of 248,841 source‑target pairs covering both cross‑category and within‑category transitions, and show that AffectDelta outperforms six baselines in affective alignment and content preservation.

By Xingzu Zhan, Lin Gu, Ruogu Fang
Hugging Face Trending Papers
Aug 11

E$^3$mo-Bench: A Scalable Benchmark for Multimodal Evoked and Expressed Emotion Understanding via Bayesian Pairwise Alignment

Understanding both expressed and evoked emotions is critical for multimodal large language models (MLLMs) to achieve comprehensive affect-aware interactions. However, existing benchmarks typically examine expressed and evoked emotions in isolation or are constrained to coarse-grained and incomplete affective characterizations.

arXiv AI
Jul 3

ESC: Emotional Self-Correction for Reliable Vision-Language Models

arXiv:2607. 02089v1 Announce Type: cross Abstract: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, yet they remain vulnerable to unreliable reasoning.

By Tien-Huy Nguyen, Minh-Nhat Nguyen, Nguyen Nhat Huy, Hung Viet Nguyen, Huy Nguyen Minh Nhat, Thanh-Huy Nguyen, Cuong Tuan Nguyen, Hoang M. Le, Dat Nguyen, Phat Kim Huynh, Min Xu, Ulas Bagci