Drive‑HWM introduces a hierarchical slow‑fast world modeling framework for autonomous driving. The slow model predicts multi‑step future representations, while the fast model jointly predicts the next frame and immediate action using a lightweight multimodal backbone and an autoregressive expert. Dynamic‑Aware Latents, learned through optical‑flow prediction, explicitly capture motion dynamics, and experiments on NAVSIM v1 and v2 show strong driving performance with validated ablation studies.
By Zhaoxin Fan, Tianbao Zhang, Wenjun Wu, Xiaofeng Wang, Yeying Jin, Jian Zhao, Zheng Zhu, Shuicheng Yan
arXiv:2610.09444v1 Announce Type: new
Abstract: Forecasting future 3D occupancy from surround-view cameras is essential for autonomous driving, yet existing approaches rely on dense voxel or bird's-e...
By Hwanhee Jung, SeungHyeon Kim, Inkyu Koo, Qixing Huang, Sang Ho Yoon, Sangpil Kim
VGGTWorld-VLA extends the VGGT-World model by adding intention-conditioned control to 3D world evolution for autonomous driving. It introduces an action–semantic conditioning mechanism that injects driving semantics and ego‑motion into the future‑token stream, allowing different future geometry predictions for the same observed scene under alternative actions. A geometry–language–action bridge further integrates historical geometry, VLA semantic features, and maneuver/trajectory representations to jointly condition future geometry prediction, achieving competitive performance on NAVSIM and demonstrating the benefits of semantic and action conditioning.
By Zhaoyang Liu, Kun Jiang, Ziying Song, Diange Yang
arXiv:2605.25059v4 Announce Type: replace
Abstract: Crucial for autonomous exploration, online 3D occupancy prediction and mapping incrementally construct dense spatial representations on the fly. Em...
By Ruoyu Wang, Yong Liu, Jiahan Li, Sheng Tao, Yuhang Lin, Yukai Ma
HERMES++ is a unified driving world model that simultaneously performs 3D scene understanding and future geometry prediction. It uses a BEV representation to merge multi‑view spatial data for LLM compatibility, introduces LLM‑enhanced world queries for knowledge transfer, and employs a Current‑to‑Future Link to condition future geometry on semantic context. A Joint Geometric Optimization strategy enforces structural integrity by combining explicit geometric constraints with implicit latent regularization, and extensive benchmark tests show that HERMES++ outperforms specialist methods in both point cloud prediction and scene understanding.
By Xin Zhou, Dingkang Liang, Xiwu Chen, Feiyang Tan, Dingyuan Zhang, Hengshuang Zhao, Xiang Bai
Camera-only 4D occupancy forecasting enables autonomous vehicles to predict future 3D semantic scenes solely from historical multi-view images, which is critical for driving safety. Even though current methods have achieved good performance, the strong spatial-temporal modeling between the input multi-view frames is still underexplored, which limits the performance of those methods in future 4D forecasting.
RiskWorld is a risk‑aware world modeling framework that forecasts shared occupancy and selectively replaces planned trajectories in automated driving. It fuses spatial risk fields, temporal actor context, and visual bird’s‑eye‑view features, using flow‑guided evolution to transport occupancy and signed residuals to correct it. In open‑loop planning on nuScenes, RiskWorld achieves the lowest collision rate over a 3‑second horizon and the second‑best average L2 error, running at 11.5 FPS on a single NVIDIA RTX 4090.
By Rongxiang Zeng, Linsen Cai, Jiafu Zhang, Yijie Zhong, Yide Tao, Shuai Wang, Nan Zheng, Hai L. Vu, Alvaro Garcia Hernandez, Yongqi Dong
As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations.
arXiv:2611.08128v1 Announce Type: new
Abstract: Current 4D occupancy world models use heavy cross-attention or deep recurrence to disentangle ego-motion from scene dynamics. This requires significant...
By Zeping Zhang, Zhuoya Zhao, Samy Metari, Robert Lagani\`ere
DriveWLA is a unified World‑Language‑Action model that integrates semantic understanding, future scene dynamics, and trajectory generation for autonomous driving. It employs a Mixture‑of‑Transformers architecture with three experts—World, Language, and Action—connected by Cross‑Expert Attention to align semantic and dynamic representations. After reinforcement learning refinement, DriveWLA outperforms existing VLM‑based and world‑model‑based planners on NAVSIM benchmarks and generalizes well to other datasets without fine‑tuning.
By Shuzheng Wang, Haoxuan Li, Shengzhen Li, Chenxi Wu, Yajing Sun, Lu Hou, Wenyu Liu, Xinggang Wang
arXiv:2606. 03159v1 Announce Type: cross Abstract: As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck.
By NVIDIA, :, Aarti Basant, Amlan Kar, Despoina Paschalidou, Fangyin Wei, Francesco Ferroni, Guillermo Garcia Cobo, Haithem Turki, Huan Ling, Jaewoo Seo, James Lucas, Jay Zhangjie Wu, Jialiang Wang, Jonathan Lorraine, Jun Gao, Kai He, Katarina Tothova, Kevin Xie, Micha{\l} Tyszkiewicz, Qi Wu, Riccardo de Lutio, Ruilong Li, Sanja Fidler, Seung Wook Kim, Tianchang Shen, Tianshi Cao, Tobias Pfaff, William Lew, Xindi Wu, Xuanchi Ren, Yifan Lu, Yuxuan Zhang, Zan Gojcic, Zian Wang
PanoBench is a new camera‑centric benchmark for panoptic occupancy prediction that unifies 3D geometry, semantics, and instance identities across synthetic and real‑world driving scenes. It combines three datasets—PanoBench‑CarlaOcc, PanoBench‑nuScenes, and PanoBench‑KITTI360—through canonical interfaces and unified evaluation protocols, providing multi‑resolution annotations with persistent instance identities. The benchmark introduces a novel reconstruction pipeline for real‑world data, a unified valid‑mask protocol, and analyses of observation fidelity, geometric completeness, spatial continuity, and temporal consistency, while also benchmarking methods on semantic, panoptic, and 4D occupancy forecasting.
whyItMatters":"PanoBench addresses the scarcity of instance‑aware annotations in panoptic occupancy research by offering comprehensive, high‑quality annotations and evaluation protocols that enable more robust development and comparison of autonomous driving perception methods."
By Yi Feng, Zizhan Guo, Junwu E, Yu Ma, Mengtan Zhang, Jin Wu, Wei Ye, Hanli Wang, Qijun Chen, Danwei Wang, Rui Fan