arXiv:2610.28625v1 Announce Type: new Abstract: Understanding human needs and selecting responses require distinct judgments that aggregate accuracy can obscure. We introduce PsycheMME, a video bench...
arXiv:2604.01569v2 Announce Type: replace Abstract: Video multimodal large language models achieve strong results on existing benchmarks, but answer accuracy alone does not establish whether they can...
arXiv:2606. 08239v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made substantial advancements in video understanding, yet the reliability of their responses remains underexplored.
SocialReasonBench is a new video‑multiple‑choice QA benchmark designed to test socially grounded reasoning in interactive narrative videos. It uses branching gameplay footage from *Detroit: Become Human*, where player choices create alternative social outcomes that can be verified against the game’s script and flowchart. The benchmark includes seven reasoning dimensions—such as intent recognition, emotional empathy, moral dilemma, counterfactual reasoning, and causal antecedent—and employs a multi‑agent pipeline to curate clips, ground answer labels, and generate theory‑guided questions with diagnostic distractors.
Omni-modal models can answer questions about video, but their predictions rarely show which participant, event, or interval supports an answer. Moreover, answer-level supervision alone does not teach...
LF-VideoQA is a new dataset for long‑form, open‑ended question answering over narrative videos, built around theory‑of‑mind questions that require reasoning across scenes averaging 23 minutes. It contains over 6,300 question‑answer‑rubric triples generated automatically from annotated screenplays and validated by humans. The dataset challenges current vision‑language models, showing that even oracle models fall short and that visual evidence is essential for answering these questions.
arXiv:2606. 04806v1 Announce Type: cross Abstract: LLMs and agentic systems are increasingly deployed in social environments, making normative competence critical for safe and appropriate behavior.
Training Multimodal Large Language Models for audio-visual social understanding is a crucial step toward embodied social intelligence. Chain-of-thought (CoT) reasoning has become the dominant approach, with HumanOmniV2 and its IntentBench benchmark as a prominent reference point.
EvaOmni is a new evidence‑grounded benchmark for audio‑visual reasoning, comprising 392 videos and 1,027 human‑annotated question‑answer pairs that span 15 capabilities and three difficulty levels. The benchmark requires models to provide both an answer and the supporting audio‑visual evidence intervals, and introduces Coverage Grounded (C.G.) and Bottleneck Grounded (B.G.) scores to evaluate evidence identification and temporal localization. Across 24 Omni LLMs, Gemini3.7‑Flash achieved the highest overall accuracy, yet its B.G. scores reveal significant room for improvement in grounded reasoning.
arXiv:2611.07836v1 Announce Type: cross Abstract: Omni-modal models can answer questions about video, but their predictions rarely show which participant, event, or interval supports an answer. Moreo...
The paper introduces ReTurn, a benchmark of 7,000 multimodal tasks designed to evaluate how models selectively use conversational history. ReTurn includes pairs that test task-carrying history (Reconfirm/Reground) and evidence-carrying history (Retrieve/Rebind), each preserving the same target question, media, and answer. Experiments on 13 vision‑language and audio‑language models show a drop in open‑ended accuracy from 93.7% to 72.3% when history is involved, revealing that high question recall does not guarantee effective task application and that competing media can misdirect responses.
arXiv:2602. 13602v2 Announce Type: replace-cross Abstract: We present \revise (\underline{Re}asoning with \underline{Vi}deo \underline{S}parsity), a multi-round agent for video question answering (VQA).