arXiv AI
Jul 1
AdaJEPA: An Adaptive Latent World Model
arXiv:2606. 32026v1 Announce Type: cross Abstract: Latent world models enable planning from high-dimensional observations by predicting future states in a compact latent space.
JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts
JEPA‑TTT is a method that continuously adapts the latent dynamics predictor of a pretrained Joint‑Embedding Predictive Architecture (JEPA) world model during test time. It performs self‑supervised updates across episodes while keeping the visual encoder and reward head fixed, using dense replay to sample prediction windows from a growing buffer. In experiments on eight dynamics shifts across four continuous‑control environments, JEPA‑TTT reduces latent prediction error by 83% and improves planning performance by 153% compared to the frozen model.
Related stories
arXiv AI
Sep 10
WorldAgen: Unified State-Action Prediction with Test-Time World Model Training
WorldAgen is a unified framework that jointly learns world modeling and action prediction using a shared Transformer backbone with two specialized heads. It introduces a Mixed Unidirectional Attention Mask to separate the world model and agent model, and enables Test-Time Training (TTT) by sampling exploratory actions and updating the world model with real state transitions. Experiments on CALVIN and LIBERO show that WorldAgen matches or surpasses state‑of‑the‑art methods, especially when TTT is applied to a few samples.
arXiv AI
4d ago
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
arXiv:2609.37250v1 Announce Type: cross Abstract: World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrai...
arXiv Machine Learning
Aug 27
JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics
JEPA-x is a cross‑predictive physics grounding method that aligns visual latent dynamics with privileged physical trajectories. By treating visual observations and physical states as two views of the same action‑conditioned trajectory and sharing a predictor, it forces the model to learn a common transition rule for both modalities. The physical branch is only used during training, so deployment incurs no extra cost, and the approach significantly reduces rollout drift and boosts control success across a multi‑task suite.
arXiv AI
4d ago
Beyond a single latent space: a dual-latent world model for long-horizon planning
The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.
arXiv AI
Jul 1
Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding
arXiv:2606. 31232v1 Announce Type: new Abstract: Learning visual world models for planning requires compact latent dynamics that remain sensitive to actions, yet reconstruction-free joint-embedding objectives can collapse to action-insensitive representations.
arXiv AI
Aug 24
WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving
arXiv:2608.20974v1 Announce Type: cross Abstract: Video Joint Embedding Predictive Architecture (V-JEPA) learns powerful spatiotemporal representations from video through self-supervised latent featu...
arXiv AI
Aug 13
Foresight Without Seeing: Latent Futures for World Action Models
arXiv:2608. 11605v1 Announce Type: new Abstract: World Action Models (WAMs) couple future visual prediction with robot action generation, enabling policies to model how the physical world evolves during interaction.
arXiv Machine Learning
Aug 26
XP-JEPA: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics
arXiv:2608.24044v1 Announce Type: new Abstract: Latent world models plan by predicting how candidate actions transform learned representations. In self-predictive models, however, the encoder and pre...
arXiv AI
Jun 29
Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
arXiv:2606. 27483v1 Announce Type: new Abstract: Large language model (LLM) agents have demonstrated strong capability in sequential decision-making, yet they remains fundamentally reactive in long-horizon tasks.
arXiv AI
2d ago
DeepJEPA: Scaling World Models from Within
DeepJEPA is a weight‑tied joint‑embedding predictive world model that treats transition depth as an inner test‑time scaling axis, learning when additional recurrent updates are worthwhile for each candidate and rollout step. Unlike traditional planners that uniformly deepen every transition, DeepJEPA concentrates extra computation on decision‑critical events such as contact onset and sustained object interaction, achieving comparable or better performance with only 1.00–1.26 updates per transition across five visual‑control settings. The approach demonstrates that improved planning does not require uniformly better object‑state decodability, but rather targeted internal computation where it can alter the planner’s elite set and action selection.
arXiv AI
Aug 3
Auto-JEPA: A Latent World Model of Continuous Intent for End-to-End Autonomous Driving
arXiv:2607. 29031v1 Announce Type: cross Abstract: Existing autonomous-driving world models typically perform dense prediction of future videos, occupancy states, BEV representations, or agent motion.