arXiv Computer Vision

CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning

arXiv AI
Aug 11

CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Models

arXiv:2608. 07621v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have recently achieved impressive performance for end-to-end autonomous driving, yet existing approaches are primarily designed for an individual single autonomous driving agent with limited support for cooperative perception, reasoning, and planning.

By Hsu-kuang Chiu, Stephen F. Smith
arXiv Computer Vision
Sep 2

Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving

Qwen-Drive-1.0 is a vision‑language foundation model tailored for autonomous driving that builds on a pretrained VLM architecture. It incorporates a bird’s‑eye‑view perception head for 3D object detection, semantic occupancy prediction, and BEV map segmentation, and a Planning Expert that generates future ego trajectories from shared representations. Experiments show strong 3D perception, driving scene understanding, and competitive motion‑planning performance while largely preserving general vision‑language capabilities.

By Xin Zhou, Zongchuang Zhao, Zhibo Yang, Mingsheng Li, Humen Zhong, Shuai Bai, Du Chu, Ruizhe Chen, Zhaohai Li, Jun Tang, Qiuyue Wang, Mingkun Yang, Jiazhao Zhang, Dayiheng Liu, Dingkang Liang, Xiang Bai
arXiv Computer Vision
Sep 21

VeriFuse: Bounded Vision-Language Arbitration and Reason-Guided Refinement for Cooperative 3D Perception

VeriFuse is a bounded arbitration framework that integrates vision‑language models (VLMs) into vehicle‑infrastructure cooperative 3D perception. Each agent first generates independent detections, then VeriFuse creates a unified candidate pool of geometric proposals and cross‑source hypotheses. A frozen VLM selects among three actions—SELECT, REFINE, or REJECT—to resolve ambiguity and produce final 3D detections, achieving strong AP50/AP70 scores on the DAIR‑V2X dataset while keeping vehicle‑side BEV AP50 drop minimal under delay.

By Hongyi Lin, Yiyao Liu, Qi Kang, Heye Huang, Yang Liu, Haris Koutsopoulos, Jinhua Zhao
arXiv AI
Jun 2

From Segments to Scenes: Temporal Understanding in Autonomous Driving via Vision-Language Model

arXiv:2512. 05277v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.

By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
arXiv Computer Vision
6d ago

Generative Scenario Rollouts for End-to-End Autonomous Driving

Generative Scenario Rollouts (GeRo) is a plug‑and‑play framework that augments Vision‑Language‑Action models for autonomous driving by jointly planning and generating language‑grounded future traffic scenes through an autoregressive rollout strategy. The approach trains a VLA model to encode vehicle and agent dynamics into latent tokens, then uses language‑conditioned autoregressive generation to produce future latent tokens and textual responses, with a rollout‑consistency loss to maintain temporal coherence and text‑action alignment. On the Bench2Drive benchmark, GeRo improves driving score by +15.7 and success rate by +26.2, and achieves state‑of‑the‑art closed‑loop and open‑loop performance, demonstrating strong zero‑shot robustness.

By Rajeev Yasarla, Deepti Hegde, Shizhong Han, Hsin-Pai Cheng, Yunxiao Shi, Meysam Sadeghigooghari, Shweta Mahajan, Apratim Bhattacharyya, Litian Liu, Risheek Garrepalli, Thomas Svantesson, Fatih Porikli, Mohammad Ghavamzadeh, Hong Cai
arXiv AI
Oct 9

More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models

The paper introduces DriveMind, a large-scale driving Visual Question Answering corpus that aligns chain-of-thought reasoning with trajectory planning. Experiments with Vision‑Language Model agents show a causal disconnect: removing navigation priors significantly harms planning, while removing the chain-of-thought has little effect. The authors propose the Reasoning‑Planning Decoupling Hypothesis and present a training‑free probe to assess an agent’s reliance on priors.

By Xurui Song, Shuo Huai, JingJing Jiang, Jiayi Kong, Jun Luo
arXiv AI
Jun 4

From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models

arXiv:2512. 05277v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.

By Kevin Cannons, Saeed Ranjbar Alvar, Mohammad Asiful Hossain, Ahmad Rezaei, Mohsen Gholami, Alireza Heidarikhazaei, Zhou Weimin, Yong Zhang, Mohammad Akbari
arXiv AI
3d ago

Can MLLMs Reason over Surround-View Driving Videos? A Multi-View Autonomous Driving QA Benchmark

The paper introduces MVDriveBench, a benchmark for evaluating Multimodal Large Language Models (MLLMs) on surround‑view driving video understanding. It features synchronized multi‑view inputs, spatio‑temporal reasoning tasks, and 12 QA categories spanning perception to decision‑making. Experiments on 23 MLLMs show limited surround‑view reasoning, and closed‑loop tests link better reasoning to improved driving performance.

By Shunming Liu, Jianyi Peng, Cheng Li, Lu Hou, Pei Huang, Yajing Sun, Huaxiao Liu