arXiv Computer Vision
PsycheMME: A Psychology-Informed Benchmark for Social Understanding and Interaction in Videos
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
Hugging Face Trending Papers
Oct 20
PsycheMME: A Psychology-Informed Benchmark for Social Understanding and Interaction in Videos
Understanding human needs and selecting responses require distinct judgments that aggregate accuracy can obscure. We introduce PsycheMME, a video benchmark informed by psychology, with eight capabilit...
arXiv Computer Vision
Oct 8
VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification
arXiv:2604.01569v2 Announce Type: replace Abstract: Video multimodal large language models achieve strong results on existing benchmarks, but answer accuracy alone does not establish whether they can...
arXiv AI
Jun 9
When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding
arXiv:2606. 08239v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made substantial advancements in video understanding, yet the reliability of their responses remains underexplored.
arXiv Computation and Language
Sep 1
SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videos
SocialReasonBench is a new video‑multiple‑choice QA benchmark designed to test socially grounded reasoning in interactive narrative videos. It uses branching gameplay footage from *Detroit: Become Human*, where player choices create alternative social outcomes that can be verified against the game’s script and flowchart. The benchmark includes seven reasoning dimensions—such as intent recognition, emotional empathy, moral dilemma, counterfactual reasoning, and causal antecedent—and employs a multi‑agent pipeline to curate clips, ground answer labels, and generate theory‑guided questions with diagnostic distractors.
arXiv Computation and Language
3d ago
LF-VideoQA: Evaluating Long-Video Understanding with Long-Form Theory-of-Mind Questions
LF-VideoQA is a new dataset for long‑form, open‑ended question answering over narrative videos, built around theory‑of‑mind questions that require reasoning across scenes averaging 23 minutes. It contains over 6,300 question‑answer‑rubric triples generated automatically from annotated screenplays and validated by humans. The dataset challenges current vision‑language models, showing that even oracle models fall short and that visual evidence is essential for answering these questions.
arXiv Computer Vision
Oct 21
EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning
EvaOmni is a new evidence‑grounded benchmark for audio‑visual reasoning, comprising 392 videos and 1,027 human‑annotated question‑answer pairs that span 15 capabilities and three difficulty levels. The benchmark requires models to provide both an answer and the supporting audio‑visual evidence intervals, and introduces Coverage Grounded (C.G.) and Bottleneck Grounded (B.G.) scores to evaluate evidence identification and temporal localization. Across 24 Omni LLMs, Gemini3.7‑Flash achieved the highest overall accuracy, yet its B.G. scores reveal significant room for improvement in grounded reasoning.