arXiv Computer Vision September 16, 2026 By Rwiddhi Chakraborty (Oliver), Yinong (Oliver), Wang, Cheng Zhang, Fan Bai, Zhuoran You, Michael Kampffmeyer, Yong Jae Lee, Fernando De la Torre, Robert Jenssen

Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models

Read the original on arXiv Computer Vision →

The paper introduces three new vision‑centric evaluation benchmarks—temporal frame retrieval, video future prediction, and causal memory distortion—to assess visual question answering in large video models. Unlike traditional benchmarks that rely on text-based multiple choice questions, these tasks require models to reason directly from visual inputs. The authors find that current state‑of‑the‑art models struggle with visual queries, highlighting a gap in visual understanding that future research should address.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Jun 9

See More, Think Deeper: Query-Expanded Visual Evidence and Answer-Clue Guided Reflection for Long Video Understanding

arXiv:2606. 09064v1 Announce Type: cross Abstract: Recent advances in Video Large Language Models (Video-LLMs) have enabled performance on long-video understanding tasks.

By Shuning Wang, Zhiheng Wu, YiNuo Lu, Naiming Liu, Chen Jia, Bowen Liu, Shuo Nie, Weijie Zhu, Yumeng Zhang

llms benchmarks

Hugging Face Trending Papers

Jun 29

From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA

High benchmark accuracy does not guarantee genuine use of visual evidence. We study this problem in traffic accident Video Question Answering (VideoQA), where correct answers should depend on scene-specific visual evidence but may instead be inferred from textual shortcuts.

llms nlp multimodal benchmarks safety

arXiv Computer Vision

Sep 10

Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering

arXiv:2609.09973v1 Announce Type: new Abstract: Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generate...

By Zizhen Wang, Bo Feng, Zhengfeng Lai, Shiyu Li, Yang Lu, Meng Cao, Ping Huang, Xiaoming Simon Wang

llms nlp benchmarks

arXiv AI

Oct 1

VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes

arXiv:2605.26380v2 Announce Type: replace-cross Abstract: Frontier multimodal large language models (MLLMs) have been reported to achieve over 90\% accuracy on fine-grained perception benchmarks. How...

By Jingru Chen, Yiming Liu, Mingtao Chen, Sijie Chen, Richeng Xuan, Liang Yang, Zhichao Hu, Fanyang Lu

llms multimodal benchmarks

arXiv Computer Vision

5d ago

Diagnosing and Correcting Bias in MLLM for Long Video Understanding

The paper introduces S‑MME, a benchmark for diagnosing shortcut‑biased behavior in Multimodal Large Language Models (MLLMs) on long‑video question answering. It proposes CLEAR, a training‑free framework that estimates and mitigates bias by treating the full video‑question pair as a complete view and intervening on hidden states during inference. Experiments on five benchmarks demonstrate that CLEAR improves prediction accuracy and robustness across different MLLMs.

By Xusheng Liang, Jianqiao Sun, Hao Zhang, Yulei Niu, Hengshuang Zhao, Jiawei Ma

llms nlp efficiency multimodal benchmarks safety

arXiv AI

Jul 14

Evidence-Backed Video Question Answering

arXiv:2607. 11862v1 Announce Type: cross Abstract: Current Video Large Language Models (Video LLMs) excel in question answering (QA) but largely operate as black boxes, providing textual answers without verifiable visual grounding.

By Shijie Wang, Honglu Zhou, Ziyang Wang, Ran Xu, Caiming Xiong, Silvio Savarese, Chen Sun, Juan Carlos Niebles

llms computer-vision nlp fine-tuning benchmarks

One email a morning, machine-written

Related stories

See More, Think Deeper: Query-Expanded Visual Evidence and Answer-Clue Guided Reflection for Long Video Understanding

From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA

Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering

VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes

Diagnosing and Correcting Bias in MLLM for Long Video Understanding

Evidence-Backed Video Question Answering