Reinforced Planning with Latent World Models introduces RP1, a neural planner that learns to evaluate imagined outcomes via a critic and improve multi‑step plans through an optimizer trained offline on world‑model roll‑outs. Unlike existing planners that are hand‑designed or only inform policies, RP1 fully learns to refine plans and can be attached to any pretrained latent world model. In experiments on visual navigation, arm reaching, and robotic manipulation, RP1 outperforms hand‑designed search algorithms, achieving near‑perfect success while using 1,000× fewer roll‑outs and up to 67× faster inference.
The paper investigates Joint-Embedding Predictive World Models (JEPA-WMs), a class of methods that perform planning in a learned representation space rather than raw input space. It systematically studies how model architecture, training objectives, and planning algorithms influence success across simulated and real‑world robotic tasks, and proposes a JEPA-WM variant that surpasses established baselines in navigation and manipulation. The authors provide code, data, and checkpoints for reproducibility.
By Basile Terver, Tsung-Yen Yang, Jean Ponce, Adrien Bardes, Yann LeCun
arXiv:2604. 03208v2 Announce Type: replace Abstract: World models are a promising path to zero-shot embodied control through planning.
By Wancong Zhang, Basile Terver, Artem Zholus, Soham Chitnis, Harsh Sutaria, Mido Assran, Randall Balestriero, Amir Bar, Adrien Bardes, Yann LeCun, Nicolas Ballas
arXiv:2605. 08732v2 Announce Type: replace-cross Abstract: Modern vision-based world models can represent observations as compact yet expressive latent manifolds, but fast goal-oriented planning in these spaces remains challenging.
By Hoang Nguyen, Xiaohao Xu, Xiaonan Huang
arXiv:2606. 07974v1 Announce Type: cross Abstract: A learned world model provides a powerful physical intuition for evaluating future states.
By Yuhai Wang, Jiawei Xia, Rongxuan Zhou, Xiao Hu, Yongliang Shi, Jing Du, Yang Ye
arXiv:2606. 01626v1 Announce Type: new Abstract: Planning with a learned latent world model is a promising route to control from raw pixels, but a strong world model alone is not enough.
By Baoqi Gao, Ruize Han, Miao Wang, Song Wang