arXiv:2511.20022v3 Announce Type: replace-cross
Abstract: Recent advancements in multimodal large language models (MLLMs) have shown strong understanding of driving scenes, drawing interest in their...
By Seungjun Yu, Seonho Lee, Namho Kim, Jaeyo Shin, Junsung Park, Wonjeong Ryu, Raehyuk Jung, Hyunjung Shim
arXiv:2512. 05277v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.
By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
arXiv:2512. 05277v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.
By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
The paper introduces DSRS, a new task that requires multimodal large language models to understand driving scenes, interpret implicit instructions, infer spatial relations, detect risks, and apply traffic rules to produce accurate segmentation masks. To support this, the authors create a benchmark dataset and propose DSRS‑Net, an architecture that integrates geometric features with multimodal fusion to improve reasoning segmentation for automated driving. Experiments show DSRS‑Net outperforms existing methods on driving scene reasoning segmentation and other language‑driven segmentation tasks, and the authors plan to release the code and dataset.
By Sicen Guo, Zhengfei Song, Kuanhsun Chen, Xu Huang, Ping Zhong, Wei Ye, Shaoyi Du, Rui Fan
arXiv:2606. 14010v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have shown strong potential for end-to-end autonomous driving by jointly modeling visual perception, language reasoning, explainability and action prediction.
By Xiangyu Huang, Zhenlin Hua, Han Zhou, Shounak Sural, Ragunathan Rajkumar
nuReasoning is a large‑scale, real‑world dataset and benchmark designed to advance reasoning‑centric autonomous driving in long‑tail scenarios. It contains 20,000 twenty‑second clips from multiple cities, each annotated with synchronized multi‑camera images, LiDAR, HD maps, object labels, and human‑verified reasoning labels covering spatial, decision, and counterfactual reasoning. The benchmark supports both reasoning evaluation and planning evaluation, demonstrating that fine‑tuning vision‑language models on nuReasoning improves question answering and that reasoning supervision boosts planning performance even without textual outputs at inference time.
By Zhiyu Huang, Johnson Liu, Rui Song, Xuewei Qi, Yiluan Guo, Zewei Zhou, Hanyin Zhang, Mingxuan Gao, Valeria Xu, Jiali Chen, Yun Zhang, Tianhui Cai, Ruining Yang, Yishan Shen, Jiaqi Ma
arXiv:2608.28762v1 Announce Type: new
Abstract: Recent advances in visual question answering (VQA) and multimodal large language models (MLLMs) have enabled natural-language reasoning over traffic sc...
By Shaozu Ding, Linan Song, Dajiang Suo
arXiv:2607. 08745v1 Announce Type: new Abstract: Recent advances in Vision-Language Models, Large Language Models, and Multimodal Large Language Models have improved autonomous driving tasks such as scene understanding, decision making, trajectory prediction, and visual question answering.
By Siddharth Damodharan, Radhika Gupta, Ali Alshami, Ryan Rabinowitz, Jugal Kalita
STRIDE (SpatioTemporal Reasoning In Driving Edge Cases) is a new benchmark designed to evaluate vision-language models on complex, safety-critical driving scenarios. It includes six task families—spatial perception, spatial understanding, temporal memory, temporal extrapolation, trajectory prediction, and scene-context awareness—using multiple-choice questions and free-form visual question answering that require detailed scene explanations and driving decision justifications. The benchmark focuses on unusual object behaviors and complex traffic interactions, and results show that even the best model, GPT‑6‑Astra, scores only 43.4% on average, highlighting significant gaps in spatiotemporal reasoning.
By Lieqi Liu, Rui gao, Jia-Chen Gu, Wenbo Hu, Zhaobin Mo, Ahmadreza Moradipari, Nejib Ammar, Wei Wang, Nanyun Peng
Existing reasoning segmentation frameworks typically employ multimodal large language models (MLLMs) with broad world knowledge to extract semantics in a human-like manner and generate corresponding s...
Recent multimodal large language models (MLLMs) have shown strong potential for autonomous driving scene understanding, yet existing methods still face a fundamental trade-off between temporal reasoning and spatial precision. Models that rely on single-frame or low-resolution inputs often miss small, distant, or partially occluded hazards, while language-centric driving models frequently provide limited grounded evidence for their explanations.
arXiv:2606. 15749v1 Announce Type: cross Abstract: Traffic scene understanding requires models to reason beyond object recognition, including lane topology, multi-view geometry, temporal evolution, and signal-phase semantics.
By Maonan Wang, Zhengyan Huang, Kemou Jiang, Yuhang Fu, Jiayue Zhu, Yuxin Cai, Xingchen Zou, Qiaosheng Zhang, Yi Yu, Ding Wang, Xi Chen, Ben M. Chen, Yuxuan Liang, Zhiyong Cui, Man On Pun, Yirong Chen