The paper introduces a low‑cost, open experimental platform for end‑to‑end autonomous driving on miniature Ackermann vehicles. It combines a physical vehicle, a printed urban track, data collection tools, trajectory registration, and a Webots digital twin to link simulation and real‑world experiments. Using command‑conditioned behavior cloning, the authors demonstrate that a neural policy can follow lanes with a mean cross‑track error of 6.1 cm, and that synthetic data plus a sim‑to‑real image translator improves performance on all track routes.
MILER is an end‑to‑end reinforcement learning framework that achieves zero‑shot sim‑to‑real transfer for autonomous driving in unstructured environments. It uses a custom semantic mid‑level representation (MLR) simulator for offline training, and during deployment it processes real camera and LiDAR data with BEVFusion to produce a compatible bird’s‑eye‑view representation. The policy’s actions are applied via a trajectory‑alignment strategy, allowing the system to drive 17.3 km on a 3.0 km test track without human intervention, all running on a Jetson AGX Orin.
By Thomas Steinecker, Denis Trescher, Alexander Bienemann, Thorsten Luettel, Mirko Maehlisch
The paper presents a compact multimodal policy trained via behavioral cloning to drive autonomously in the CARLA simulator. Using five‑frame histories of RGB images, LiDAR, telemetry, and lane waypoints, the 1.36‑million‑parameter model predicts throttle, brake, and steering at 20 Hz. Trained on 236,882 windows (≈3.3 hours of driving) from 448 captures, the policy drives for hours on both training and unseen routes without collisions, demonstrating qualitative transfer and recovery from large trajectory deviations.
By Jordy Kieto
OPTED is a method for on‑policy fine‑tuning of end‑to‑end driving models that separates reinforcement learning from the policy update. A privileged teacher trained with RL on vectorized inputs (HD‑maps and bounding boxes) supervises the pre‑trained student during closed‑loop post‑training. Applied to the camera‑based models TransFuser and VaVAM in AlpaSim, OPTED boosts driving scores by 1.6× and 9.5×, respectively, while requiring roughly three orders of magnitude fewer simulator interactions than direct RL post‑training.
By Damiano Da Col, Maximilian Igl, Peter Karkus, Kashyap Chitta, Boris Ivanovic, Marco Pavone, Konrad Schindler, Christos Sakaridis
arXiv:2604. 03497v2 Announce Type: replace-cross Abstract: Vision-language-model (VLM)-guided reinforcement learning (RL) has recently attracted significant attention for it, replacing brittle hand-crafted rewards with semantically grounded signals; however, deploying such simulation-trained policies on real vehicles remains a fundamental challenge, because they rely on simulator-native observations and simulator-coupled action semantics with no counterpart on physical hardware.
By Zilin Huang, Zhengyang Wan, Zihao Sheng, Boyue Wang, Junwei You, Sikai Chen
arXiv:2607. 13028v1 Announce Type: cross Abstract: Training robust autonomous driving agents requires a simulator that is fast enough for reinforcement learning at scale, realistic enough to ground behavior in real-world map structure, and diverse enough to cover the safety-critical long tail that logged data rarely contains.
By Zhouchonghao Wu, Akshay Rangesh, Weixin Li, Wei-Jer Chang, Zachary Lee, Tim Wang, Wei Zhan
arXiv:2606. 17386v1 Announce Type: cross Abstract: End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments.
By Zikang Xiong, Weixin Li, Zhouchonghao Wu, Akshay Rangesh, Saarth Bonde, Grantland Hall, Chen Tang, Yihan Hu, Wei Zhan
arXiv:2607. 20785v1 Announce Type: cross Abstract: Deploying navigation systems at scale requires a recipe that minimizes sensor assumptions, generalizes across robot embodiments, and trains efficiently.
By Arjun Majumdar, Avinash Sooriyarachchi, Benjamin Tibi, Chris Bamford, Elliot Chane-Sane, Guillaume Lample, Khyathi Raghavi Chandu, Ludovic Ho Fuh, Mathieu Poiree, Olivier Duchenne, Rosalie Millner, Srijan Mishra, Theo Cachet, Thomas Chabal
arXiv:2608. 19425v1 Announce Type: cross Abstract: Reliable performance evaluation is a central bottleneck for deploying robot-learning policies in real-world conditions.
By Dijie Zhu, Seunghun Oh, Ruopeng Huang, Zhiyu Huang, Jiaqi Ma, Chen Tang
arXiv:2606. 03159v1 Announce Type: cross Abstract: As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck.
By NVIDIA, :, Aarti Basant, Amlan Kar, Despoina Paschalidou, Fangyin Wei, Francesco Ferroni, Guillermo Garcia Cobo, Haithem Turki, Huan Ling, Jaewoo Seo, James Lucas, Jay Zhangjie Wu, Jialiang Wang, Jonathan Lorraine, Jun Gao, Kai He, Katarina Tothova, Kevin Xie, Micha{\l} Tyszkiewicz, Qi Wu, Riccardo de Lutio, Ruilong Li, Sanja Fidler, Seung Wook Kim, Tianchang Shen, Tianshi Cao, Tobias Pfaff, William Lew, Xindi Wu, Xuanchi Ren, Yifan Lu, Yuxuan Zhang, Zan Gojcic, Zian Wang
arXiv:2608.22301v1 Announce Type: cross
Abstract: Humans imitate at the level of intent: given a demonstration, we infer its goal and carry it out with whatever tools, objects, and layouts are at han...
By Xunzhe Zhou, Yiyang Cai, Fengyi Wang, Ran Ju, Hanxiang Ren, Ruizhe Liu, Yu Zhang, Qian Luo, Feng Chen, Pei Zhou, Yi Ma, Yanchao Yang
As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations.