arXiv Computer Vision

Time-to-Goal: Learned Temporal Metrics for Long-Horizon Planning with JEPA World Models

arXiv AI
Sep 30

Beyond a single latent space: a dual-latent world model for long-horizon planning

The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.

By Delin Zhao, Zhengrong Yue, Shaobin Zhuang, Junlin He, Xiaoyu Chen, Zikang Wang, Yuxin Liu, Limin Wang, Yali Wang
arXiv Machine Learning
Oct 9

How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models

The paper introduces TEMPO, a temporal-distance planning objective that augments latent world models by learning a map of the frozen latent space where distances correspond to the number of environment steps between states. TEMPO blends this learned temporal distance into the planner’s cost, requiring no additional rewards or policies, and adds negligible computational overhead. Experiments on eleven simulated environments show that TEMPO consistently improves planner performance across all goal distances, notably raising LeWM’s success rate from 36% to 99% on the TwoRoom task.

By Lama Moukheiber, Haotian Xue, Yongxin Chen
arXiv AI
Oct 19

Beyond Action Prediction: Learning Search-Free Planning in Latent Space

The paper introduces Hindsight Inverse JEPA Planning (HIJP), a search‑free framework that replaces costly online action sequence optimization with a goal‑conditioned inverse model. HIJP trains by relabeling future observations as goals, providing action supervision across multiple horizons and shaping visual representations. The method also aligns predicted actions with a latent predictor model to enforce action‑transition consistency, achieving high success rates and significant speedups on visual control tasks.

By Guanchen Du, Jinhao Du, Yanpeng Sun, Shan Zhang, Na Zhao, Zechao Li
arXiv AI
Aug 17

Traj-LeWM: Path-Aware World-Model Planning via Latent Trajectory Cost

arXiv:2608. 14125v1 Announce Type: new Abstract: LeWM is a lightweight visual world model that learns latent dynamics end-to-end from pixels and ranks candidate action sequences by the distance between their predicted endpoints and the goal.

By Xiaodi Huang, Ziyi Ding, Jingtian Wan, Yuchen Liu, Yuan Zhang, Xiao-Ping Zhang, Jiayu Chen, Zhang Zhang, Tao Huang
arXiv Machine Learning
Sep 30

FlexiWorld: Learning and Planning via Flexible Action Chunks Across Multiple Time Scales

FlexiWorld is a JEPA-based latent world model that learns variable‑length action chunks across multiple time scales for goal‑directed planning. It jointly trains a causal action encoder and an autoregressive actor, using mixed‑span goal supervision and Student Forcing to reduce exposure bias. In experiments on four benchmarks, FlexiWorld with the Actor‑Residual Cross‑Entropy Method (ARCEM) achieves higher mean success rates than the strongest baseline and supports flexible planning chunk lengths without retraining.

By Shidu Ren, Qilin Gu, Zhenghao Ni, Junhan Sun, Jiaqi Wang, Damien Scieur, Yunze Liu
arXiv Machine Learning
Oct 7

Reperesentation Geometry Matters for Planning with JEPA World Models

The paper introduces SCALE (State-CAlibrated Latent Embeddings), a technique that aligns pairwise latent distances with task-relevant state-space distances in joint-embedding predictive world models. By adding SCALE to the LeWorldModel (LeWM) objective, the authors preserve LeWM’s architecture while ensuring that latent geometry reflects meaningful task outcomes. Experiments demonstrate that SCALE improves planning success across manipulation and navigation tasks with various solvers, and the authors analyze how the method reshapes representation geometry to support planning.

By Jiaming Hu, Yan Zheng, Shi Bo, Tian Wang, Florian Dubost, Alejandro Mottini, Junze Liu, Arvind Srinivasan, Kai Zhong, Kun Qian, Sharon Gao, Qingjun Cui