arXiv Computer Vision By Shiva Sreeram, Erfan Aasi, Tsun-Hsuan Wang, Alaa Maalouf, Daniela Rus

Mind the Temporal Gap: Evaluating Video Models for Temporal Understanding in Driving

Read the original on arXiv Computer Vision →

The paper investigates whether modern video‑language models can reliably understand temporal dynamics in driving scenarios. Using controlled ego‑motion tasks, the authors find that various prompting strategies are inconsistent and heavily biased, while newer models only show modest few‑shot improvements. However, lightweight LoRA fine‑tuning and probes on intermediate visual features can recover strong motion discrimination, suggesting that motion cues exist within the models but are not effectively leveraged by the final video‑to‑language pathway.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Jun 2

From Segments to Scenes: Temporal Understanding in Autonomous Driving via Vision-Language Model

arXiv:2512. 05277v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.

By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
arXiv AI
Jun 4

From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models

arXiv:2512. 05277v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.

By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
arXiv AI
Oct 19

STRIDE: Evaluating Spatiotemporal Reasoning in Driving Edge Cases

STRIDE (SpatioTemporal Reasoning In Driving Edge Cases) is a new benchmark designed to evaluate vision-language models on complex, safety-critical driving scenarios. It includes six task families—spatial perception, spatial understanding, temporal memory, temporal extrapolation, trajectory prediction, and scene-context awareness—using multiple-choice questions and free-form visual question answering that require detailed scene explanations and driving decision justifications. The benchmark focuses on unusual object behaviors and complex traffic interactions, and results show that even the best model, GPT‑6‑Astra, scores only 43.4% on average, highlighting significant gaps in spatiotemporal reasoning.

By Lieqi Liu, Rui gao, Jia-Chen Gu, Wenbo Hu, Zhaobin Mo, Ahmadreza Moradipari, Nejib Ammar, Wei Wang, Nanyun Peng
arXiv AI
Jun 2

Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events

arXiv:2606. 02522v1 Announce Type: cross Abstract: Video multimodal large language models (MLLMs) have made rapid progress on general and long-form video understanding, yet their ability to preserve brief answer-critical visual evidence remains underexplored.

By Xiaolin Liu, Yilun Zhu, Xiangyu Zhao, Xuehui Wang, Yan Li, Xin Li, Haoyu Cao, Xing Sun, Shaofeng Zhang, Xu Yang, Zhihang Zhong, Xue Yang
arXiv AI
Sep 2

TempCloze: Can Video-LLMs Identify the Missing Middle?

TempCloze is a video cloze benchmark designed to evaluate visual temporal reasoning in Video-LLMs. The task presents a video’s beginning and ending clips and asks models to select the correct missing middle from four candidates, focusing on semantic, alignment, and progression aspects while minimizing appearance cues. Evaluation of 31 models shows that temporal alignment is the main challenge, with models performing better on semantic content and event progression but struggling to place events correctly in time.

By Wenqi Pei, Henry Hengyuan Zhao, Yilai Liu, Jiahao Meng, Han Chen, Ziyu Wang, Hongyang Du
arXiv Computer Vision
Sep 1

Improving Spatial-Temporal Reasoning in Video-Language Models with Structured Video Prompting

The paper introduces structured video prompting, a training‑free inference‑time technique that augments input videos with lightweight spatial and temporal structure to provide explicit anchors for evidence organization. By applying this method to two video benchmarks and two open video‑language models, the authors demonstrate performance improvements across several tasks, with gains varying by model and task. The study suggests that failures in video‑language models stem not only from reasoning capacity but also from how video evidence is presented during inference.

By Sadegh Mohammadian