arXiv Computer Vision

HybridWorldSim: A Scalable and Controllable High-fidelity Simulator for Autonomous Driving

arXiv Machine Learning
Jun 3

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset

arXiv:2606. 02956v1 Announce Type: cross Abstract: Existing autonomous driving datasets have enabled major progress, but fall short in sensor fidelity, map completeness, or geographic diversity.

By Richard Schwarzkopf, Fabian Immel, Alexander Blumberg, Jonas Merkert, Nils Rack, Kaiwen Wang, Fabian Konstantinidis, Julian Truetsch, Carlos Fernandez, Annika B\"atz, Kevin R\"osch, Marlon Steiner, Willi Poh, Yinzhe Shen, Royden Wagner, Felix Hauser, Dominik Strutz, Jaime Villa, Gleb Stepanov, Holger Caesar, \"Omer \c{S}ahin Ta\c{s}, Frank Bieder, Jan-Hendrik Pauls, Christoph Stiller
arXiv Computer Vision
Sep 25

HelloWorld: Towards Practical Applications of Generative Driving World Models

HelloWorld is a 2B-parameter driving world model that learns from diverse video data to generate controllable driving scenarios. It uses ego pose, HD maps, and 3D boxes to produce coherent multi‑sensor outputs, including synchronized seven‑camera RGB and conditional LiDAR. The system is designed for efficient repeated inference and is evaluated on visual quality, control fidelity, cross‑view consistency, robustness, and inference speed.

By Fan Lu, Hanshi Wang, Zijing Wang, Quan Feng, Zhi Wang, Shijie Chen, Xianming Zeng, Yujian Zhang, Jiazhe Wang, Xin Zha, Kai Wang, Zhijie Zhao, Lin Zhu, Tianyi Yang, Yucheng Xu, Tao Ji, Haodong Zhang, Zhipeng Zhang, Peixi Peng, Guang Chen, Xingliang Liu, Lei Yang, Jianyun Xu
arXiv AI
Oct 19

DriveWorld3D Evidence Constrained 3D World Modeling and Generative Simulation for Real-World Autonomous Driving

DriveWorld3D is a framework that integrates temporal video learning, multi‑sensor measurement constraints, and offline generative simulation to build a unified 3D world model for autonomous driving. Its online component tracks road surfaces, volumetric occupancy, dynamic entities, and road relations while explicitly managing physical occupancy, observability, and evidence provenance. The offline component reconstructs scenes with neural radiance fields or 3D Gaussian Splatting and expands rare conditions using generative models, but limits generated content from being used as geometric ground truth.

By Wei Wang (Robostreet research team Cambridge, MA, USA), Yiru Veronika Wang (Robostreet research team Cambridge, MA, USA), Yihan Wang (Robostreet research team Cambridge, MA, USA), Zihao Li (Robostreet research team Cambridge, MA, USA), Xiaohui Liang (Robostreet research team Cambridge, MA, USA)
arXiv AI
Jun 3

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

arXiv:2606. 03159v1 Announce Type: cross Abstract: As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck.

By NVIDIA, :, Aarti Basant, Amlan Kar, Despoina Paschalidou, Fangyin Wei, Francesco Ferroni, Guillermo Garcia Cobo, Haithem Turki, Huan Ling, Jaewoo Seo, James Lucas, Jay Zhangjie Wu, Jialiang Wang, Jonathan Lorraine, Jun Gao, Kai He, Katarina Tothova, Kevin Xie, Micha{\l} Tyszkiewicz, Qi Wu, Riccardo de Lutio, Ruilong Li, Sanja Fidler, Seung Wook Kim, Tianchang Shen, Tianshi Cao, Tobias Pfaff, William Lew, Xindi Wu, Xuanchi Ren, Yifan Lu, Yuxuan Zhang, Zan Gojcic, Zian Wang
Hugging Face Trending Papers
Jun 2

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations.

arXiv AI
Sep 28

NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigation

NavGen introduces a text-to-video data generation pipeline that creates about 400K vision‑language navigation episodes for both indoor and outdoor scenes, using high‑fidelity visual generative models. The approach includes a style‑diversification method to scale up rare, hard‑to‑collect data. Models trained on NavGen data outperform those trained on existing UAV navigation datasets and achieve a 75% success rate in real‑world flying experiments.

By Xijie Huang, Yongyang Wan, Chengbin Dong, Zimo Ding, Mo Zhu, Yijin Wang, Zhiyang Liu, Fei Gao, Yuze Wu, Xin Zhou
arXiv Computer Vision
Oct 21

HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

HERMES++ is a unified driving world model that simultaneously performs 3D scene understanding and future geometry prediction. It uses a BEV representation to merge multi‑view spatial data for LLM compatibility, introduces LLM‑enhanced world queries for knowledge transfer, and employs a Current‑to‑Future Link to condition future geometry on semantic context. A Joint Geometric Optimization strategy enforces structural integrity by combining explicit geometric constraints with implicit latent regularization, and extensive benchmark tests show that HERMES++ outperforms specialist methods in both point cloud prediction and scene understanding.

By Xin Zhou, Dingkang Liang, Xiwu Chen, Feiyang Tan, Dingyuan Zhang, Hengshuang Zhao, Xiang Bai
arXiv Computer Vision
Aug 25

BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generation

arXiv:2608.22187v1 Announce Type: cross Abstract: Modern driving action models are increasingly improved in a self-improvement loop, where a learned world simulator imagines future observations and t...

By Jiaqi Wang, Zhuo Zhang, Haining Guan, Tingguang Zhou, Haowen Cui, Zhongyang Zhu, Yulong Zheng, ChuanYe Wang, Xuefeng Chen, Zhen Yang, Tianchen Deng, Feiyang Tan, Hangning Zhou, Bo Dai, Lixia Shen, Xiwu Chen, Xiyang Wang, Jiajun Zhu