arXiv Computer Vision

ForeDrive: Foresight-Guided End-to-End Autonomous Driving with a Planning-Relevant Latent World Model

ForeDrive introduces a planning-relevant latent world model that is asymmetrically coupled to a Diffusion Transformer planner. The model learns multi‑horizon latent futures with a JEPA‑style world model, while planning gradients update the shared encoder and stop‑gradient routing trains the predictor with forecasting losses only. Gated visual fusion, future‑status injection, and Trajectory‑Adaptive Bias are used to guide trajectory generation without overriding current observations, achieving high performance on NAVSIM benchmarks using only front‑view images and pure imitation learning.

arXiv Machine Learning
Oct 9

PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving

PlanWAM introduces a Planning‑Shaped World Action Model that tailors future‑state representations to be most useful for trajectory planning in end‑to‑end autonomous driving. It compresses historical observations with a Temporal Register Pyramid, shapes a future latent representation using a privileged posterior branch guided by planning objectives, and distills this into a prior branch that predicts the representation from history alone. The resulting foresighted planning context yields state‑of‑the‑art performance on NAVSIM and HUGSIM benchmarks.

By Jinchang Xu, Hongda Yu, Fengwei Dong, Wenhui Huang, Xi Wei, Yongzhi Liu, Sunan Zhang, Jirao Wang, Chen Lv, Bingbing Li, Guodong Yin, Weichao Zhuang
arXiv Computer Vision
Sep 28

WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

WALT introduces a method to align latent trajectories with pretrained driving world models, creating a compact generative trajectory space that preserves action-relevant semantics without altering the original model. The approach uses a dual-branch autoencoder to map raw waypoints into this latent space and transfers visual world knowledge into trajectory representations. Experiments on NAVSIM benchmarks show modest performance gains and a 30.5% reduction in planner FLOPs, indicating that maintaining world representations while extracting action-relevant information can improve trajectory planning efficiency.

By Mingkai Jia, Jiaxin Guo, Zhijian Shu, Jiawei Xu, Mingxiao Li, Jintao Cheng, Ping Tan, Wei Yin
arXiv AI
Aug 24

WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving

arXiv:2608.20974v1 Announce Type: cross Abstract: Video Joint Embedding Predictive Architecture (V-JEPA) learns powerful spatiotemporal representations from video through self-supervised latent featu...

By Xinlin Wang, Yujiao Xiang, Yuheng Zhou, Jingqi Wang, Minqing Huang, Jiajie Huang, Dongxu Wei, Tingguang Zhou, Xiyang Wang, Gong Chen, Zhi Xu, Feiyang Tan, Hangning Zhou, Mu Yang
arXiv AI
Aug 20

DA-WAM: Decision-Aligned Future Latents for Driving World Models

DA‑WAM is a framework that integrates predictive representation learning, action‑conditioned future modeling, and trajectory scoring into a single decision‑making objective for autonomous driving. It uses an online encoder with a stable momentum target to keep future representations aligned with the driving task, generating a distinct future latent for each trajectory candidate. A future‑latent‑conditioned scorer evaluates these latents, with expert‑matched trajectories supervised by observed futures and safety‑critical hard negatives providing additional guidance, achieving state‑of‑the‑art results on NAVSIM‑v1 and NAVSIM‑v2.

By Ruiguo Zhong, Benshan Ma, Xiaolong Chen, Lang Zhang, Mingyue Feng, Yaonong Wang, Pei Liu, Jun Ma
arXiv AI
Oct 19

Beyond Action Prediction: Learning Search-Free Planning in Latent Space

The paper introduces Hindsight Inverse JEPA Planning (HIJP), a search‑free framework that replaces costly online action sequence optimization with a goal‑conditioned inverse model. HIJP trains by relabeling future observations as goals, providing action supervision across multiple horizons and shaping visual representations. The method also aligns predicted actions with a latent predictor model to enforce action‑transition consistency, achieving high success rates and significant speedups on visual control tasks.

By Guanchen Du, Jinhao Du, Yanpeng Sun, Shan Zhang, Na Zhao, Zechao Li
arXiv AI
Jun 30

X-Mind: Efficient Visual Chain-of-Thought via Predictive World Model for End-to-End Driving

arXiv:2606. 28758v1 Announce Type: cross Abstract: Predicting future states is essential for autonomous agents, yet current Vision-Language-Action (VLA) models fundamentally lack this capability, relying instead on reactive perception-action mapping.

By Bohao Zhao, Chengrui Wei, Guangfeng Jiang, Ruixin Liu, Xuejie Lv, Liu Liang, Sutao Deng, Xiuyang Fan, Pengkun Zheng, Jinyun Zhou, Rui Guo, Hanpeng Liu, Yutong Zheng, Yi Guo, Xinlong Zheng, Qingyu Luo, Zhuangzhuang Ding, Yu Zhang, Hang Zhang, Xianming Liu
arXiv Computer Vision
Oct 27

DriveUniTok: A Unified Multimodal Tokenizer for Driving World-Action Models

arXiv:2610.35044v1 Announce Type: new Abstract: World models are emerging as a fundamental component of policy learning for real-world agents, enabling agents to predict future world states and plan...

By Jingbo Xu, Hanchao Leng, JiaHui Huang, Hanbing Li, Jingwei Zhao, Kuiyuan Yang, Chen Gao, Guang Chen, Hangjun Ye, Yong Li