arXiv AI

Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving

Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving proposes EMPlan, a hybrid trajectory planning method that combines sparse anchors with an offset refinement module for low-latency, high-accuracy predictions. The approach uses a two-stage training paradigm—pretraining followed by reward-guided fine-tuning—to improve safety without extra inference cost, leveraging rule-based reward signals and unpaired preference supervision. EMPlan is evaluated on the non-reactive NAVSIM benchmark, achieving a favorable balance between planning accuracy and efficiency under real-time constraints.

arXiv AI

4d ago

Evaluation Is All You Need for Multi-Modal Autonomous Driving

The paper introduces iDriveVLA, a multi‑modal planning framework for autonomous driving that addresses a generation‑evaluation asymmetry by improving candidate trajectory spaces and providing a unified, safety‑aware evaluator. It combines a Safety‑aware Scorer for risk estimation with a VLM‑guided Modulator that adapts weighting to the scene, and employs an oracle‑aligned progressive training strategy. On the NAVSIM v1 leaderboard, iDriveVLA achieves a new state‑of‑the‑art PDMS score of 94.95, surpassing human‑expert performance.

By Zeyu He, Shiqi Liu, Ke Chen, Yun Yan, Jinzi Wu, Dianqiao Lei, Sirui Wang, ShuRui Peng, Tao Chen, Zhuo Huang, Yu Wu, Yadong Shao, Zhichao Li, Ke Sun, Yang Guan, Keqiang Li, Shengbo Eben Li

agents robotics multimodal benchmarks safety

arXiv AI

Aug 11

OmniV2X: A Generative Foundation Planner for Efficient End-to-End Cooperative Driving

arXiv:2606. 21165v2 Announce Type: replace-cross Abstract: We present OmniV2X, a generative foundation model for vehicle-to-everything (V2X) cooperative driving.

By Juntong Peng, Juanwu Lu, Yupeng Zhou, Can Cui, Yaobin Chen, Ziran Wang

agents fine-tuning multimodal benchmarks

arXiv AI

Jul 24

Drive As You Like: Multi-Head Diffusion with Reinforcement Learning for Personalized Driving

arXiv:2508. 16947v2 Announce Type: replace-cross Abstract: Despite significant progress, imitation learning-based autonomous driving planners remain largely restricted to reproducing high-frequency biased behaviors, overlooking the inherent behavioral diversity of human driving.

By Fan Ding, Xuewen Luo, Fucai Ke, Hwa Hui Tew, Susilawati Susilawati, Vishnu Monn Baskaran, Junn Yong Loo

llms agents diffusion reinforcement-learning robotics benchmarks safety

arXiv AI

Jun 24

FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning

arXiv:2606. 24231v1 Announce Type: new Abstract: Multimodal driving planning faces a long-standing tension between two paradigms: scoring-based methods benefit from dense reward supervision but are confined to a fixed action vocabulary, while anchor-based methods generate proposals dynamically yet suffer from sparse supervision constrained to a single ground-truth trajectory.

By Xirui Li, Zhe Liu, Xiaoqing Ye, Wenhua Han, Yifeng Pan, Junyu Han, Hengshuang Zhao

multimodal benchmarks safety

arXiv AI

Sep 15

Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMs

arXiv:2609.15322v1 Announce Type: cross Abstract: Pretrained driving vision-language models (VLMs) integrate visual, route, language, and driving context into rich driving priors, yet their represent...

By Changxin Lu, Xiaoliang Meng, Yu Wu, Rui Huang, Honglin Li, Tao Chen, Kaixuan Zhou, Yadong Shao

llms multimodal

arXiv Computer Vision

Aug 28

SimWAM: A Simple World Action Model for End-to-End Autonomous Driving

SimWAM is a lightweight World-Action Model that uses future‑video prediction only during training to supervise an action expert, enabling end‑to‑end autonomous driving without costly test‑time future imagination. The architecture co‑trains a pretrained video expert and a lightweight action expert via joint flow matching, while an isolated attention mask keeps action prediction independent of future frames. This design allows the video backbone to be swapped and the action expert scaled independently, achieving 91.5 PDMS on NAVSIM, outperforming state‑of‑the‑art WAM planners with lower latency and zero‑shot transfer to nuScenes.

By Zongchuang Zhao, Xin Zhou, Tianyang Xu, Zhengyang Sun, Kaixuan Zhou, Yu Wu, Honglin Li, Dingkang Liang, Xiang Bai

agents diffusion reinforcement-learning robotics benchmarks

arXiv AI

Jun 11

ConsistencyPlanner: Real-time Planning with Fast-Sampling Consistency Models

arXiv:2606. 11569v1 Announce Type: cross Abstract: Closed-loop planning in complex, real-world driving scenarios presents a critical challenge for autonomous driving systems.

By Qichao Zhang, Xing Fang, Jiaqi Fang, Zhenwen Cai, Jie Ling, Qiankun Yu, Dongbin Zhao

agents robotics multimodal safety

arXiv AI

Jul 7

AnchorVLA: Bridging Discrete Decisions and Continuous Trajectories for Vision-Language-Action Planning

arXiv:2607. 03182v1 Announce Type: cross Abstract: Autonomous driving planning requires translating navigation intent, traffic rules, dynamic interactions, and language instructions into executable continuous trajectories.

By Qi Liu, Yabei Li, Hongsong Wang, Heng Zhang, Lei He

llms agents robotics multimodal benchmarks safety

arXiv AI

Sep 1

Aligning Multi-Trajectory Supervision with Policy Optimization for VLA Driving

arXiv:2608.30122v1 Announce Type: cross Abstract: Vision-language-action (VLA) driving methods increasingly combine multi-trajectory imitation learning with group-relative policy optimization (GRPO),...

By Tian Zhang, Zhuo Huang, Hongrui Ye, Yu Wu, Zengmao Wang, Kaixuan Zhou

reinforcement-learning efficiency multimodal safety

arXiv AI

Jun 6

CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving

arXiv:2606. 06219v1 Announce Type: cross Abstract: End-to-end autonomous driving models often struggle to balance multi-modal maneuver generation with real-time inference constraints.

By Yining Xing, Zehong Ke, Zhiyuan Liu, Yanbo Jiang, Wenhao Yu, Jianqiang Wang

agents diffusion robotics fine-tuning multimodal benchmarks safety

arXiv Computer Vision

2d ago

AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving

arXiv:2601.01762v4 Announce Type: replace-cross Abstract: Practical autonomous driving requires models that generalize by reasoning through spatial-temporal possibilities to exclude unsafe outcomes....

By Yanhao Wu, Haoyang Zhang, Fei He, Rui Wu, Yanhu Shan, Congpei Qiu, Liang Gao, Wei Ke, Tong Zhang

agents robotics benchmarks safety

arXiv Machine Learning

Jul 31

S-GRPO: Unified Post-Training for Large Vision-Language Models

arXiv:2604. 16557v2 Announce Type: replace Abstract: Current post-training methodologies for adapting Large Vision-Language Models (LVLMs) generally fall into two paradigms: Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL).

By Yuming Yan, Kai Tang, Sihong Chen, Ke Xu, Dan Hu, Qun Yu, Pengfei Hu

llms reinforcement-learning fine-tuning multimodal

Related stories