arXiv Computer Vision September 18, 2026 By Sethuraman T V, Savya Khosla, Aditi Tiwari, Vidya Ganesh, Rakshana Jayaprakash, Aditya Jain, Vignesh Srinivasakumar, Onkar Kishor Susladkar, Srinidhi Sunkara, Aditya Shanmugham, Rakesh Vaideeswaran, Abbaas Alif Mohamed Nishar, Simon Jenni, Rohan Maheshwari, Derek Hoiem

Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models

Read the original on arXiv Computer Vision →

The paper introduces REVEAL, a diagnostic benchmark that stresses Video‑Language Models (VidLMs) on five controlled probes—camera‑motion sensitivity, cross‑frame integration, video sycophancy, language‑only shortcuts, and temporal expectation bias—to assess how well these models encode and use visual evidence. Experiments on 12 VidLMs reveal systematic failures: some visual signals are never reliably encoded, while others are overridden by model priors, leading to performance below chance on several probes that humans solve with high accuracy. Mechanistic probes further pinpoint where and why visual evidence is lost, demonstrating that under assertive prompts a model’s output becomes nearly invariant to real versus random video input, rendering visual evidence causally inert.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Jun 2

Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events

arXiv:2606. 02522v1 Announce Type: cross Abstract: Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.

By Xiaolin Liu, Yilun Zhu, Xiangyu Zhao, Xuehui Wang, Yan Li, Xin Li, Haoyu Cao, Xing Sun, Shaofeng Zhang, Xu Yang, Zhihang Zhong, Xue Yang

llms multimodal benchmarks

arXiv AI

Jul 16

Accuracy Without Grounding: Diagnosing Visual Dependency Dissociation in Video LLM Benchmarks

arXiv:2607. 13305v1 Announce Type: cross Abstract: Benchmark accuracy in video large language models (LLMs) is often treated as evidence of visual understanding.

By Jae Joong Lee

llms benchmarks

arXiv Machine Learning

1d ago

SYNCR: Diagnosing and Learning Cross-Video Reasoning from Simulation

arXiv:2609.37918v1 Announce Type: cross Abstract: Reasoning across videos requires aligning events, matching identities, comparing motion, and integrating partial observations. Evaluating these capab...

By Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami

llms fine-tuning multimodal benchmarks

arXiv AI

Sep 2

TempCloze: Can Video-LLMs Identify the Missing Middle?

TempCloze is a video cloze benchmark designed to evaluate visual temporal reasoning in Video-LLMs. The task presents a video’s beginning and ending clips and asks models to select the correct missing middle from four candidates, focusing on semantic, alignment, and progression aspects while minimizing appearance cues. Evaluation of 31 models shows that temporal alignment is the main challenge, with models performing better on semantic content and event progression but struggling to place events correctly in time.

By Wenqi Pei, Henry Hengyuan Zhao, Yilai Liu, Jiahao Meng, Han Chen, Ziyu Wang, Hongyang Du

llms benchmarks safety

arXiv AI

Aug 14

TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint

arXiv:2608. 13167v1 Announce Type: cross Abstract: When visual evidence is occluded or chaotic, models should abstain.

By Fnu Pramono, John Cai, Sourabh Kulkarni

llms multimodal benchmarks

arXiv AI

Sep 18

AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models

AVTrace is a diagnostic suite designed to evaluate audio‑visual temporal reasoning in omni models. It covers tasks such as onset and span grounding, synchronization, next‑step prediction, cross‑modal localization, chain parsing, and event‑conditioned comprehension, providing 34,114 training examples and balanced development and test splits. Five open omni models were tested, all scoring below the majority‑label baseline on synchronization verification and showing low performance on chain parsing and event‑conditioned tasks, while parameter‑efficient temporal post‑training improved some metrics.

By Longyin Zhang, Parth Sakhare Mahendra, Chengwei Wei, Ning Zhang, Lim Ming Chong, Sirui He, Ai Ti Aw

multimodal benchmarks

One email a morning, machine-written

Related stories

Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events

Accuracy Without Grounding: Diagnosing Visual Dependency Dissociation in Video LLM Benchmarks

SYNCR: Diagnosing and Learning Cross-Video Reasoning from Simulation

TempCloze: Can Video-LLMs Identify the Missing Middle?

TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint

AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models