arXiv AI
PI-MLLM: Psychophysics-Inspired Multimodal Large Language Model for Oddly Satisfying Video Emotion Analysis
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
2d ago
CLEAR: Coupling Local Evidence with Affective Representations for Multimodal Emotion Recognition in Conversation
arXiv:2605.18547v2 Announce Type: replace Abstract: In video calls and livestreamed discussions, recognizing speakers' emotions helps identify emotionally salient segments and supports fine-grained v...
arXiv Computer Vision
Aug 28
HUG-VIS: A Multimodal Benchmark for Human-centered Understanding and Generation in Visual Intelligence
HUG‑VIS is a unified multimodal benchmark for human‑centered visual intelligence, comprising 8,400 half‑body videos of 30 professional actors performing 280 emotion‑action prompts in Mandarin. The dataset provides synchronized video, audio, text, and alpha mattes for four tasks—human emotion recognition, video generation, voice cloning, and video matting—allowing evaluation of both open‑ and closed‑source models under a zero‑shot protocol. Results reveal that linguistic cues dominate emotion recognition, visual affect is weakest, and that automatic metrics and human judgments diverge in generation and cloning tasks, while motion‑related boundary fidelity remains a key challenge for matting.
Hugging Face Trending Papers
Aug 11
E$^3$mo-Bench: A Scalable Benchmark for Multimodal Evoked and Expressed Emotion Understanding via Bayesian Pairwise Alignment
Understanding both expressed and evoked emotions is critical for multimodal large language models (MLLMs) to achieve comprehensive affect-aware interactions. However, existing benchmarks typically examine expressed and evoked emotions in isolation or are constrained to coarse-grained and incomplete affective characterizations.
arXiv Machine Learning
Jun 3
Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals
arXiv:2606. 02679v1 Announce Type: new Abstract: Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed.
arXiv AI
Jun 9
Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An Evaluation
arXiv:2606. 07541v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have shown strong performance on objective tasks such as video understanding and reasoning.
arXiv AI
Jun 2
Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
arXiv:2606. 02522v1 Announce Type: cross Abstract: Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.