TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations
arXiv:2606. 17386v1 Announce Type: cross Abstract: End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments.
arXiv:2606. 17386v1 Announce Type: cross Abstract: End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments.
arXiv:2608. 04964v1 Announce Type: new Abstract: Interactive video world models are essential for long-horizon planning and exploration, yet they suffer from compounding errors.
arXiv:2602. 13977v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) promises to unlock capabilities beyond imitation learning for Vision--Language--Action (VLA) models, but its requirement for massive real-world interaction prevents direct deployment on physical robots.
MAPLE is a new framework that enables reactive, multi‑agent rollout of dynamic driving scenarios directly in the latent space of vision‑language‑action models. It trains the ego vehicle and surrounding traffic agents independently over multiple steps, using supervised fine‑tuning followed by reinforcement learning with safety, progress, and interaction rewards. The approach is scalable, does not rely on external simulators, and achieves state‑of‑the‑art performance on Bench2Drive.
TerraTransfer++ demonstrates a demonstration‑free, end‑to‑end autonomous driving framework that leverages self‑play in vectorized simulators to pretrain a policy. The method distills self‑play rollouts into a flow‑matching trajectory teacher and trains lightweight adapters on a frozen vision backbone, using action KL divergence, a batch‑relational low‑rank loss, and trajectory‑matching loss. The resulting closed‑loop policy surpasses all published imitation‑based methods on the HUGSim benchmark, while the planner achieves state‑of‑the‑art performance on NAVSIM navhard and navtest without any human demonstrations.
SV-WAM is a surround‑view world‑action model that keeps all six camera views while enabling efficient inference by discarding the video branch at deployment. It uses future‑video prediction as dense training supervision and an action‑centered causal mask to prevent action tokens from attending to future‑video tokens during joint denoising. A differentiable drivable‑area compliance regularizer penalizes vehicle‑footprint corners near or crossing drivable boundaries, improving safety and boundary awareness. Experiments on NAVSIMv2 and nuScenes show state‑of‑the‑art planning performance with low latency and strong zero‑shot transfer.
End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all existing paradigms: imi...
The paper introduces a reinforcement learning post‑training scheme that trains robot world models on their own autoregressive rollouts, using a contrastive RL objective adapted from diffusion models. It also proposes a training protocol that compares multiple variable‑length futures, a multi‑view visual fidelity reward, and demonstrates state‑of‑the‑art rollout fidelity on the DROID dataset, outperforming baselines on LPIPS, SSIM, and human preference tests.
SV-WAM is a surround‑view world‑action model that keeps all six camera views for autonomous driving while enabling efficient inference by discarding the video branch during deployment. It uses future‑video prediction as dense training supervision and introduces an action‑centered causal mask to prevent future‑video tokens from influencing action tokens during joint denoising. A differentiable drivable‑area compliance regularizer further improves safety by penalizing vehicle‑footprint corners that approach or cross drivable boundaries. Experiments on NAVSIMv2 and nuScenes show state‑of‑the‑art planning performance with low latency and strong zero‑shot transfer.
DriveWLA is a unified World‑Language‑Action model that integrates semantic understanding, future scene dynamics, and trajectory generation for autonomous driving. It employs a Mixture‑of‑Transformers architecture with three experts—World, Language, and Action—connected by Cross‑Expert Attention to align semantic and dynamic representations. After reinforcement learning refinement, DriveWLA outperforms existing VLM‑based and world‑model‑based planners on NAVSIM benchmarks and generalizes well to other datasets without fine‑tuning.
Generative Scenario Rollouts (GeRo) is a plug‑and‑play framework that augments Vision‑Language‑Action models for autonomous driving by jointly planning and generating language‑grounded future traffic scenes through an autoregressive rollout strategy. The approach trains a VLA model to encode vehicle and agent dynamics into latent tokens, then uses language‑conditioned autoregressive generation to produce future latent tokens and textual responses, with a rollout‑consistency loss to maintain temporal coherence and text‑action alignment. On the Bench2Drive benchmark, GeRo improves driving score by +15.7 and success rate by +26.2, and achieves state‑of‑the‑art closed‑loop and open‑loop performance, demonstrating strong zero‑shot robustness.
arXiv:2608.24885v1 Announce Type: cross Abstract: Action-conditioned world models are increasingly used as learned simulators for policy evaluation and improvement, yet their effectiveness rests on a...