The paper introduces the concept of decision‑metric alignment, which ensures that Euclidean distance to a goal latent in JEPA‑style latent world models correctly ranks action sequences for model‑predictive control. It proposes two metrics—Plan‑Real Spearman and CEM‑stage Spearman—to evaluate latent–real rank agreement, and identifies encoder distortion, terminal rollout error, and candidate margins as key factors affecting alignment. Building on these insights, the authors present DA‑LeWM, an enhanced latent world model that incorporates inverse‑dynamics and demonstration‑conditioned goal‑action heads, leading to faster convergence and higher online success rates compared to the baseline LeWM while maintaining similar probe scores.
By Jiawei Wang, Ke Rui, Yushen Zuo, Yichun Feng, Minglei Li
arXiv:2608. 14125v1 Announce Type: new Abstract: LeWM is a lightweight visual world model that learns latent dynamics end-to-end from pixels and ranks candidate action sequences by the distance between their predicted endpoints and the goal.
By Xiaodi Huang, Ziyi Ding, Jingtian Wan, Yuchen Liu, Yuan Zhang, Xiao-Ping Zhang, Jiayu Chen, Zhang Zhang, Tao Huang
arXiv:2611.05847v1 Announce Type: cross
Abstract: Joint-Embedding Predictive Architectures (JEPAs) support planning by comparing predicted latent states with an encoded goal. For this comparison to g...
By Ruilong Yu, Fei Ye
arXiv:2608. 16287v1 Announce Type: new Abstract: Joint-embedding predictive world models plan by scoring predicted terminal embeddings against a goal embedding using a cost defined on the representation itself.
By Jiaming Hu, Yan Zheng, Tian Wang
arXiv:2609.24749v1 Announce Type: cross
Abstract: Latent world models predict the consequences of actions, but accurate prediction does not guarantee that latent distance reflects which candidate wil...
By Shuaijun Liu, Chengyu Wu, Qifu Wen, Feiyang You, Chenglong Zhang, Shuyang Hao, Xi Lin, Ningxin Su
The paper introduces TEMPO, a temporal-distance planning objective that augments latent world models by learning a map of the frozen latent space where distances correspond to the number of environment steps between states. TEMPO blends this learned temporal distance into the planner’s cost, requiring no additional rewards or policies, and adds negligible computational overhead. Experiments on eleven simulated environments show that TEMPO consistently improves planner performance across all goal distances, notably raising LeWM’s success rate from 36% to 99% on the TwoRoom task.
By Lama Moukheiber, Haotian Xue, Yongxin Chen
Latent Energy Action Planning (LEAP) improves model predictive control by treating the entire action horizon as a differentiable variable and optimizing it using a frozen LeWorldModel (LeWM). LEAP couples terminal latent goal matching with a terminal-window state energy, ensuring both the predicted terminal latent and the decoder-predicted terminal descriptor align with the goal. In four control domains, LEAP raises mean success from 77.5% (LeWM+CEM) to 94.8%, a 17.3‑percentage‑point improvement while keeping the frozen LeWM representation.
arXiv:2606.20627v2 Announce Type: replace
Abstract: Joint-Embedding Predictive Architectures (JEPAs) enable agents to plan in latent space by imagining the outcomes of candidate actions, yet task spe...
By Samuel Barbeau, Simon Roy, Giovanni Beltrame, Christian Desrosiers, Nicolas Thome
The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.
By Delin Zhao, Zhengrong Yue, Shaobin Zhuang, Junlin He, Xiaoyu Chen, Zikang Wang, Yuxin Liu, Limin Wang, Yali Wang
The paper presents an end‑to‑end JEPA world model that enhances latent prediction with inverse dynamics and state alignment to improve goal‑conditioned robotic planning. By preventing latent collapse and grounding representations in physical configuration, the model achieves top success rates on tasks such as TwoRoom, PushT, and OGBench‑Cube, outperforming the baseline LeWorldModel. Ablation studies confirm that state alignment consistently boosts planning success over inverse dynamics alone across all four benchmark tasks.
By Muyuan Liu (GENISOM AI, Beijing, China), Yue Huang (GENISOM AI, Beijing, China), Zheng Liang (GENISOM AI, Beijing, China), Xiang Gao (GENISOM AI, Beijing, China)
Latent Energy Action Planning (LEAP) is a new method that treats the entire action horizon as a differentiable variable and optimizes it using a frozen LeWorldModel (LeWM). LEAP couples terminal latent goal matching with a terminal‑window state energy, ensuring that the predicted terminal latent and decoder‑predicted terminal descriptor align with the goal. Using a frozen goal‑conditioned proposal, a quasi‑Newton solver, and post‑optimization projection, LEAP improves mean success from 77.5% to 94.8% across four control domains while keeping the LeWM representation frozen.
By Phu Pham, Aniket Bera
The paper introduces SCALE (State-CAlibrated Latent Embeddings), a technique that aligns pairwise latent distances with task-relevant state-space distances in joint-embedding predictive world models. By adding SCALE to the LeWorldModel (LeWM) objective, the authors preserve LeWM’s architecture while ensuring that latent geometry reflects meaningful task outcomes. Experiments demonstrate that SCALE improves planning success across manipulation and navigation tasks with various solvers, and the authors analyze how the method reshapes representation geometry to support planning.
By Jiaming Hu, Yan Zheng, Shi Bo, Tian Wang, Florian Dubost, Alejandro Mottini, Junze Liu, Arvind Srinivasan, Kai Zhong, Kun Qian, Sharon Gao, Qingjun Cui