arXiv AI

Adapting Generalist Vehicle Models for High-Speed MPC Across Terrains

arXiv:2607. 13319v1 Announce Type: cross Abstract: High-speed off-road autonomy requires precise closed-loop control for a target vehicle while remaining robust across changing terrains.

arXiv AI
5d ago

Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization

The paper introduces Endpoint-Constrained Optimization (ECO), a lightweight postprocessing layer that corrects intermediate waypoints of end-to-end driving policies while preserving the predicted endpoint. ECO does not require maps, privileged simulator state, or additional training, and can be applied to a wide range of waypoint-emitting policies. Experiments on two closed-loop simulators show that ECO significantly improves closed-loop performance, achieving top results in the HUGSIM Closed-Loop Driving Challenge and boosting scene scores on AlpaSim.

By Brayden Zhang, Mahsa Golchoubian, Igor Gilitschenski, Boris Ivanovic, Kashyap Chitta
arXiv AI
Sep 2

CoLT-Drive: Counterfactual Long-Tail Benchmarking and Knowledge-Preserving Adaptation for Driving Affordance Prediction

The paper introduces CoLT-Drive, a 3,536-sample counterfactual long‑tail benchmark for evaluating decision‑level driving affordance prediction, which tests whether models can infer how rare objects affect an ego vehicle’s high‑level actions. It also proposes KPA, a knowledge‑preserving adaptation framework that combines structured prompting, expert merging, and a regime‑aware LoRA mixture‑of‑experts module to improve small VLMs on driving tasks. Experiments show KPA achieves 60.8% pair accuracy on CoLT‑Drive, outperforming the Qwen3‑VL‑2B baseline and LoRA SFT while keeping competitive in‑domain performance.

By Zhengxu Tang, Guofeng Cui, Ziyu Gong, Xiaozhou Zhang, Ruifeng Deng, Chengzhi Qi, Ke Chen, Sachin Patil, Tianjun Xiao, Langechuan Liu, Pichao Wang
arXiv AI
Jun 3

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

arXiv:2606. 03159v1 Announce Type: cross Abstract: As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck.

By NVIDIA, :, Aarti Basant, Amlan Kar, Despoina Paschalidou, Fangyin Wei, Francesco Ferroni, Guillermo Garcia Cobo, Haithem Turki, Huan Ling, Jaewoo Seo, James Lucas, Jay Zhangjie Wu, Jialiang Wang, Jonathan Lorraine, Jun Gao, Kai He, Katarina Tothova, Kevin Xie, Micha{\l} Tyszkiewicz, Qi Wu, Riccardo de Lutio, Ruilong Li, Sanja Fidler, Seung Wook Kim, Tianchang Shen, Tianshi Cao, Tobias Pfaff, William Lew, Xindi Wu, Xuanchi Ren, Yifan Lu, Yuxuan Zhang, Zan Gojcic, Zian Wang
arXiv AI
Sep 10

PV-WM: A Heterogeneous Micro-Macro World Model for Articulated Pedestrian-Vehicle Co-Rollout

PV-WM is a history‑only world model that jointly predicts pedestrian root motion, 15‑joint articulation, and vehicle kinematic states in a synchronized heterogeneous state. It uses recurrent updates to generate pedestrian and vehicle motion chunks, reconstructing vehicle boxes from predicted center, heading, and observed extent, and recomputes pedestrian‑vehicle geometry after each transition. Compared to a one‑shot predictor, PV‑WM reduces Root ADE by 12.7% and MPJPE by 14.8%, and across 824 Waymo contexts it lowers Root ADE by 5.2%, MPJPE by 7.6%, P‑V distance error by 11.9%, and oriented‑box closest‑approach error by 5.8%, while using 57.1% fewer parameters, 96.5% fewer FLOPs, and 25.5% lower p95 latency.

By Haozhuang Chi, Jingsong Liang, Ziying Song, Lei Yang, Shihao Li, Haoruo Zhang, Chen Lv
arXiv Computer Vision
Sep 18

Feeling Terrain Before Crossing: World Models for Off-Road Navigation

The paper introduces Feel‑WM, an off‑road navigation world model that incorporates proprioceptive data to predict both visual scenes and the robot’s physical sensations such as slip, tilt, and shake. By learning a future proprioceptive state and failure risk from the robot’s own experience, the model can evaluate planned trajectories using a separable score that balances goal similarity with predicted failure risk. Experiments on real and simulated off‑road data show that Feel‑WM outperforms visual‑only models in both open‑loop planning and closed‑loop navigation for wheeled and legged robots, and it successfully guides a Husky robot around rough terrain on mountain trails where an end‑to‑end policy fails.

By E-In Son, Dong-Wook Kim, Ji-Hoon Hwang, Kangsun Lee, Jisung Bae, Jung-Taak Kim, Seung-Woo Seo
Hugging Face Trending Papers
Jun 2

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations.

arXiv AI
Jul 15

TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale

arXiv:2607. 13028v1 Announce Type: cross Abstract: Training robust autonomous driving agents requires a simulator that is fast enough for reinforcement learning at scale, realistic enough to ground behavior in real-world map structure, and diverse enough to cover the safety-critical long tail that logged data rarely contains.

By Zhouchonghao Wu, Akshay Rangesh, Weixin Li, Wei-Jer Chang, Zachary Lee, Tim Wang, Wei Zhan
arXiv AI
Jun 10

TaCarla: A comprehensive benchmarking dataset for end-to-end autonomous driving

arXiv:2602. 23499v4 Announce Type: replace-cross Abstract: Collecting a high-quality dataset is a critical task that demands meticulous attention to detail, as overlooking certain aspects can render the entire dataset unusable.

By Tugrul Gorgulu, Atakan Dag, M. Esat Kalfaoglu, Halil Ibrahim Kuru, Baris Can Cam, Halil Ibrahim Ozturk, Ozsel Kilinc