arXiv AI

RD-JEPA: Predictive latent pretraining for few-trajectory transfer across reaction--diffusion equations

RD‑JEPA is a joint‑embedding predictive architecture designed for self‑supervised pretraining on reaction‑diffusion trajectories. The model is pretrained on five parameterized systems and then adapted to three held‑out systems that were not seen during pretraining. Using as few as one, five, or ten complete trajectories from a held‑out system, RD‑JEPA outperforms five supervised surrogate baselines, an independently trained control that removes the trajectory‑dependent predictive latent pathway, and an architecture‑matched model trained from scratch, achieving lower mean relative discrete β field error and mean absolute spatial first‑difference error across various output resolutions, forecast horizons, and adaptation trajectory choices.

arXiv AI
Jun 2

Improving Diffusion Planners by Self-Supervised Action Gating with Energies

arXiv:2603. 02650v2 Announce Type: replace-cross Abstract: Diffusion planners are a strong approach for offline reinforcement learning, but they can fail when value-guided selection favours trajectories that score well yet are locally inconsistent with the environment dynamics, resulting in brittle execution.

By Yuan Lu, Dongqi Han, Yansen Wang, Dongsheng Li
arXiv Computer Vision
Sep 23

ForeDrive: Foresight-Guided End-to-End Autonomous Driving with a Planning-Relevant Latent World Model

ForeDrive introduces a planning-relevant latent world model that is asymmetrically coupled to a Diffusion Transformer planner. The model learns multi‑horizon latent futures with a JEPA‑style world model, while planning gradients update the shared encoder and stop‑gradient routing trains the predictor with forecasting losses only. Gated visual fusion, future‑status injection, and Trajectory‑Adaptive Bias are used to guide trajectory generation without overriding current observations, achieving high performance on NAVSIM benchmarks using only front‑view images and pure imitation learning.

By Sinuo Wang, Zichong Gu, Yuhan Huang, Wenxin Wen, Xun Yang, Yiqing Zhang, Xingyu Zhang, Ningyu Che, Jie Ling, Qiankun Yu, Wei Liu, Jing Xu, Xinggang Wang
arXiv Machine Learning
Aug 27

JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics

JEPA-x is a cross‑predictive physics grounding method that aligns visual latent dynamics with privileged physical trajectories. By treating visual observations and physical states as two views of the same action‑conditioned trajectory and sharing a predictor, it forces the model to learn a common transition rule for both modalities. The physical branch is only used during training, so deployment incurs no extra cost, and the approach significantly reduces rollout drift and boosts control success across a multi‑task suite.

By Kehan Wen, Ziming Li, Siyuan Luo, Fan Shi
arXiv Machine Learning
Oct 3

JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

JEPA‑TTT is a method that continuously adapts the latent dynamics predictor of a pretrained Joint‑Embedding Predictive Architecture (JEPA) world model during test time. It performs self‑supervised updates across episodes while keeping the visual encoder and reward head fixed, using dense replay to sample prediction windows from a growing buffer. In experiments on eight dynamics shifts across four continuous‑control environments, JEPA‑TTT reduces latent prediction error by 83% and improves planning performance by 153% compared to the frozen model.

By Zheyuan Zhang, Suyu Ye, Nakul Agarwal, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Daniel Khashabi, Tianmin Shu, Vaishnav Tadiparthi
arXiv AI
Aug 3

RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving

arXiv:2602. 07339v2 Announce Type: replace Abstract: Diffusion-based trajectory planners can model multi-modal driving behavior, but their iterative denoising process introduces a latency bottleneck for real-time closed-loop deployment.

By Ruturaj Reddy, Hrishav Bakul Barua, Junn Yong Loo, Thanh Thi Nguyen, Ganesh Krishnasamy
arXiv Machine Learning
Oct 7

Identifiable World Models from Pretrained Diffusion Representations

The paper demonstrates that a frozen pretrained diffusion model can be endowed with identifiable latent coordinates via a lightweight alignment map called Contrastive Diffusion Alignment (ConDA). By leveraging auxiliary-variable nonlinear ICA guarantees, ConDA aligns diffusion latents so that the resulting representation recovers latent dynamical states up to permutation and componentwise invertible transformations, preserves the underlying structural causal model, and simplifies lagged graph recovery. Experiments on physical and robotic video systems show that ConDA, especially under TCL and GCL assumptions, achieves near-perfect state recovery and accurate lagged graph reconstruction, even in simulated falling-body and bipedal robot scenarios.

By Ruchi Sandilya, Conor Liston, Logan Grosenick
arXiv AI
Oct 9

MA-JEPA: Joint-Embedding World Models for Multi-Agent Reinforcement Learning

MA-JEPA is a stochastic world model designed for multi‑agent reinforcement learning that replaces observation reconstruction with joint‑embedding prediction. It uses a categorical latent state and a causal Transformer trained on posterior and action‑conditioned dynamics to generate imagined trajectories for actor‑critic learning. The model achieves strong performance on the SMAC benchmark, matching or surpassing the best reported mean win rates on half of the evaluated maps.

By Brandon Gary Kaplowitz, Osaze James Obahor, Christian Schroeder de Witt