arXiv AI By Shunming Liu, Jianyi Peng, Cheng Li, Lu Hou, Pei Huang, Yajing Sun, Huaxiao Liu

Can MLLMs Reason over Surround-View Driving Videos? A Multi-View Autonomous Driving QA Benchmark

Read the original on arXiv AI →

The paper introduces MVDriveBench, a benchmark for evaluating Multimodal Large Language Models (MLLMs) on surround‑view driving video understanding. It features synchronized multi‑view inputs, spatio‑temporal reasoning tasks, and 12 QA categories spanning perception to decision‑making. Experiments on 23 MLLMs show limited surround‑view reasoning, and closed‑loop tests link better reasoning to improved driving performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jun 2

From Segments to Scenes: Temporal Understanding in Autonomous Driving via Vision-Language Model

arXiv:2512. 05277v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.

By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
arXiv AI
Jun 4

From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models

arXiv:2512. 05277v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.

By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
arXiv Computer Vision
Nov 5

DSRS: Teaching MLLMs to Perceive, Reason, and Segment in Complex Driving Scenes

The paper introduces DSRS, a new task that requires multimodal large language models to understand driving scenes, interpret implicit instructions, infer spatial relations, detect risks, and apply traffic rules to produce accurate segmentation masks. To support this, the authors create a benchmark dataset and propose DSRS‑Net, an architecture that integrates geometric features with multimodal fusion to improve reasoning segmentation for automated driving. Experiments show DSRS‑Net outperforms existing methods on driving scene reasoning segmentation and other language‑driven segmentation tasks, and the authors plan to release the code and dataset.

By Sicen Guo, Zhengfei Song, Kuanhsun Chen, Xu Huang, Ping Zhong, Wei Ye, Shaoyi Du, Rui Fan
arXiv Computer Vision
Nov 6

nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving

nuReasoning is a large‑scale, real‑world dataset and benchmark designed to advance reasoning‑centric autonomous driving in long‑tail scenarios. It contains 20,000 twenty‑second clips from multiple cities, each annotated with synchronized multi‑camera images, LiDAR, HD maps, object labels, and human‑verified reasoning labels covering spatial, decision, and counterfactual reasoning. The benchmark supports both reasoning evaluation and planning evaluation, demonstrating that fine‑tuning vision‑language models on nuReasoning improves question answering and that reasoning supervision boosts planning performance even without textual outputs at inference time.

By Zhiyu Huang, Johnson Liu, Rui Song, Xuewei Qi, Yiluan Guo, Zewei Zhou, Hanyin Zhang, Mingxuan Gao, Valeria Xu, Jiali Chen, Yun Zhang, Tianhui Cai, Ruining Yang, Yishan Shen, Jiaqi Ma