The paper presents an end‑to‑end JEPA world model that enhances latent prediction with inverse dynamics and state alignment to better support goal‑conditioned robotic planning. By incorporating inverse dynamics, the model prevents latent collapse and encodes action information, while state alignment ties consecutive latent states to their physical configurations and motions. Experiments on four benchmark tasks show the model achieves top success rates on TwoRoom, PushT, and OGBench‑Cube, and its state alignment consistently improves planning performance over inverse dynamics alone.
The paper introduces Hindsight Inverse JEPA Planning (HIJP), a search‑free framework that replaces costly online action sequence optimization with a goal‑conditioned inverse model. HIJP trains by relabeling future observations as goals, providing action supervision across multiple horizons and shaping visual representations. The method also aligns predicted actions with a latent predictor model to enforce action‑transition consistency, achieving high success rates and significant speedups on visual control tasks.
By Guanchen Du, Jinhao Du, Yanpeng Sun, Shan Zhang, Na Zhao, Zechao Li
LeWAM is an end‑to‑end joint‑embedding predictive architecture that learns compact latent world models without pixel reconstruction and directly supports policy execution and planning. It trains jointly on next‑latent‑state prediction and action‑flow matching while imposing an isotropic Gaussian constraint on the latent state. The approach yields significant performance gains, raising visual‑planning success rates from 28.6 % to 89.7 % on contact‑rich tasks, from 10.9 % to 46.2 % on long‑horizon tasks, and speeding up planning by 32.3×, while remaining competitive on behavior‑cloning tasks and providing interpretable control‑relevant representations.
By Minghao Fu, Tavis Siebert, Eryk Halicki, Randall Balestriero
arXiv:2606. 31232v1 Announce Type: new Abstract: Learning visual world models for planning requires compact latent dynamics that remain sensitive to actions, yet reconstruction-free joint-embedding objectives can collapse to action-insensitive representations.
By Zhenghao Zhang, Yuanxiang Wang, Zhenyu Guan, Yujia Yang, Bingkang Shi, Tianyu Zong, Hongzhu Yi, Guoqing Chao, Xingchen Chen, Tiankun Yang, Chenxi Bao, Tao Yu, Jingjing Zhou, Jungang Xu
arXiv:2606.20627v2 Announce Type: replace
Abstract: Joint-Embedding Predictive Architectures (JEPAs) enable agents to plan in latent space by imagining the outcomes of candidate actions, yet task spe...
By Samuel Barbeau, Simon Roy, Giovanni Beltrame, Christian Desrosiers, Nicolas Thome
arXiv:2608.29434v1 Announce Type: cross
Abstract: JEPA world models make latent-space planning a practical route to control, but they are built almost exclusively on images. Whether latent prediction...
By Fabio F. Oberweger, Michael Schwingshackl
The paper introduces SCALE (State-CAlibrated Latent Embeddings), a technique that aligns pairwise latent distances with task-relevant state-space distances in joint-embedding predictive world models. By adding SCALE to the LeWorldModel (LeWM) objective, the authors preserve LeWM’s architecture while ensuring that latent geometry reflects meaningful task outcomes. Experiments demonstrate that SCALE improves planning success across manipulation and navigation tasks with various solvers, and the authors analyze how the method reshapes representation geometry to support planning.
By Jiaming Hu, Yan Zheng, Shi Bo, Tian Wang, Florian Dubost, Alejandro Mottini, Junze Liu, Arvind Srinivasan, Kai Zhong, Kun Qian, Sharon Gao, Qingjun Cui
The Representation World Model (RWM) learns states, transitions, and executable plans directly within a representation space, bypassing traditional explicit dynamics models and action-space search. It uses inverse-dynamics supervision along latent paths to shape the representation geometry, enabling direct planning by constructing a latent path between current and goal states and recovering actions via inverse dynamics. Experiments on continuous-control benchmarks and robotic manipulation tasks demonstrate RWM’s effectiveness and potential for complex embodied control.
By Yijun Yuan, Weicheng Zheng, Weibang Wang, Minghui Qin, Chang Sun, Junhao Huang, Kenan Li, Anmin Liu, Yicheng Yao, Hang Zhao
arXiv:2604. 03208v2 Announce Type: replace Abstract: World models are a promising path to zero-shot embodied control through planning.
By Wancong Zhang, Basile Terver, Artem Zholus, Soham Chitnis, Harsh Sutaria, Mido Assran, Randall Balestriero, Amir Bar, Adrien Bardes, Yann LeCun, Nicolas Ballas
The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.
By Delin Zhao, Zhengrong Yue, Shaobin Zhuang, Junlin He, Xiaoyu Chen, Zikang Wang, Yuxin Liu, Limin Wang, Yali Wang
FIRM-WM is a compact pixel world model that separates a goal‑comparable configuration from a 128‑dimensional dynamic fiber, enabling reward‑free visual planning from offline videos. It addresses two key mismatches: aligning planning states with goal images and reconciling factual trajectories with interventional sampling. In experiments, FIRM‑WM achieves high success rates on TwoRoom, Reacher, and OGBench‑Cube while using fewer parameters and faster planning times than prior models.
By Yilun Wu, Yunjian Zhang, Aobo Li, Mujiangshan Wang, Haitao Wu, Aqiang Zhang
LeWAM is a bidirectional transformer that predicts forward, backward, inverse dynamics and policy, trained end‑to‑end on a decoder‑free JEPA latent. It outperforms a forward‑only JEPA world model by better aligning latent representations with robot and object states while ignoring visual distractors. In closed‑loop tests, LeWAM matches a flow‑matching policy of the same size and improves MPC planning by sampling actions in the policy head’s noise space rather than raw actions.
By Shashank Hegde, Alexander Popov, Elie Aljalbout, Nikolai Smolyanskiy