The paper investigates how latent world models (specifically JEPA-style models) use Euclidean distance to a goal latent as a cost for model‑predictive control (MPC). It introduces two metrics—Plan‑Real Spearman and CEM‑stage Spearman—to evaluate how well latent‑space distances align with real‑task progress, a property termed decision‑metric alignment. By identifying encoder distortion, terminal rollout error, and candidate margins as key factors, the authors propose DA‑LeWM, which augments the base model with inverse‑dynamics and demonstration‑conditioned goal‑action heads, leading to faster convergence and higher online success while maintaining similar probe scores.
arXiv:2608. 16287v1 Announce Type: new Abstract: Joint-embedding predictive world models plan by scoring predicted terminal embeddings against a goal embedding using a cost defined on the representation itself.
By Jiaming Hu, Yan Zheng, Tian Wang
arXiv:2611.05847v1 Announce Type: cross
Abstract: Joint-Embedding Predictive Architectures (JEPAs) support planning by comparing predicted latent states with an encoded goal. For this comparison to g...
By Ruilong Yu, Fei Ye
arXiv:2608. 14125v1 Announce Type: new Abstract: LeWM is a lightweight visual world model that learns latent dynamics end-to-end from pixels and ranks candidate action sequences by the distance between their predicted endpoints and the goal.
By Xiaodi Huang, Ziyi Ding, Jingtian Wan, Yuchen Liu, Yuan Zhang, Xiao-Ping Zhang, Jiayu Chen, Zhang Zhang, Tao Huang
arXiv:2609.24749v1 Announce Type: cross
Abstract: Latent world models predict the consequences of actions, but accurate prediction does not guarantee that latent distance reflects which candidate wil...
By Shuaijun Liu, Chengyu Wu, Qifu Wen, Feiyang You, Chenglong Zhang, Shuyang Hao, Xi Lin, Ningxin Su
Latent Energy Action Planning (LEAP) improves model predictive control by treating the entire action horizon as a differentiable variable and optimizing it using a frozen LeWorldModel (LeWM). LEAP couples terminal latent goal matching with a terminal-window state energy, ensuring both the predicted terminal latent and the decoder-predicted terminal descriptor align with the goal. In four control domains, LEAP raises mean success from 77.5% (LeWM+CEM) to 94.8%, a 17.3‑percentage‑point improvement while keeping the frozen LeWM representation.
Latent Energy Action Planning (LEAP) is a new method that treats the entire action horizon as a differentiable variable and optimizes it using a frozen LeWorldModel (LeWM). LEAP couples terminal latent goal matching with a terminal‑window state energy, ensuring that the predicted terminal latent and decoder‑predicted terminal descriptor align with the goal. Using a frozen goal‑conditioned proposal, a quasi‑Newton solver, and post‑optimization projection, LEAP improves mean success from 77.5% to 94.8% across four control domains while keeping the LeWM representation frozen.
By Phu Pham, Aniket Bera
The paper introduces TEMPO, a temporal-distance planning objective that augments latent world models by learning a map of the frozen latent space where distances correspond to the number of environment steps between states. TEMPO blends this learned temporal distance into the planner’s cost, requiring no additional rewards or policies, and adds negligible computational overhead. Experiments on eleven simulated environments show that TEMPO consistently improves planner performance across all goal distances, notably raising LeWM’s success rate from 36% to 99% on the TwoRoom task.
By Lama Moukheiber, Haotian Xue, Yongxin Chen
arXiv:2606.20627v2 Announce Type: replace
Abstract: Joint-Embedding Predictive Architectures (JEPAs) enable agents to plan in latent space by imagining the outcomes of candidate actions, yet task spe...
By Samuel Barbeau, Simon Roy, Giovanni Beltrame, Christian Desrosiers, Nicolas Thome
AD-WM is a new action‑discriminative joint‑embedding world model designed for counterfactual model predictive control. It augments residual latent dynamics with action‑recovery regularization based on inverse dynamics and conditional mutual information, while discarding auxiliary heads at test time so that MPC remains unchanged. Experiments on OGBench‑Cube and other simulation environments show substantial gains in hard‑start success and mean success, and zero‑shot transfer to a Franka robot improves pick‑and‑place success from 42.2% to 71.1%.
By Jiabin Qiu, Zixuan Chen, Hongye Cao, Jieqi Shi, Jing Huo, Yang Gao
The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.
By Delin Zhao, Zhengrong Yue, Shaobin Zhuang, Junlin He, Xiaoyu Chen, Zikang Wang, Yuxin Liu, Limin Wang, Yali Wang
The paper presents an end‑to‑end JEPA world model that enhances latent prediction with inverse dynamics and state alignment to improve goal‑conditioned robotic planning. By preventing latent collapse and grounding representations in physical configuration, the model achieves top success rates on tasks such as TwoRoom, PushT, and OGBench‑Cube, outperforming the baseline LeWorldModel. Ablation studies confirm that state alignment consistently boosts planning success over inverse dynamics alone across all four benchmark tasks.
By Muyuan Liu (GENISOM AI, Beijing, China), Yue Huang (GENISOM AI, Beijing, China), Zheng Liang (GENISOM AI, Beijing, China), Xiang Gao (GENISOM AI, Beijing, China)