arXiv Machine Learning
Jun 18
Hierarchical Planning with Latent World Models
arXiv:2604. 03208v2 Announce Type: replace Abstract: World models are a promising path to zero-shot embodied control through planning.
Latent Goal Prediction from Language for Model-Based Planning
Related stories
arXiv AI
6d ago
Beyond a single latent space: a dual-latent world model for long-horizon planning
The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.
arXiv AI
Jun 16
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
arXiv:2606. 15768v1 Announce Type: cross Abstract: Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene.
arXiv Computer Vision
5d ago
The Planning Limits of Latent World Models
arXiv:2609.39235v1 Announce Type: cross Abstract: World models offer a promising way to help robots understand how the physical world evolves and plan complex behaviours through imagination. Yet exis...
arXiv AI
5d ago
Language-Conditioned World Modeling for Visual Navigation
arXiv:2603.26741v2 Announce Type: replace-cross Abstract: Goal-conditioned visual navigation has been a long-standing testbed for embodied AI. We study a natural language-conditioned variant, languag...
arXiv AI
Aug 17
Traj-LeWM: Path-Aware World-Model Planning via Latent Trajectory Cost
arXiv:2608. 14125v1 Announce Type: new Abstract: LeWM is a lightweight visual world model that learns latent dynamics end-to-end from pixels and ranks candidate action sequences by the distance between their predicted endpoints and the goal.
arXiv AI
Jun 17
PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space
arXiv:2606. 17924v1 Announce Type: cross Abstract: Current Vision-Language-Action (VLA) models face a trade-off between efficient action generation and explicit deliberation.
arXiv AI
5d ago
Grounded World Model: Latent Planning with Language Goals
arXiv:2604.11751v2 Announce Type: replace-cross Abstract: World models such as DINO-WM and LeWM specify the goal with an image, which is difficult to obtain in advance for novel tasks. We present the...
arXiv Computer Vision
Aug 26
LeFlow: Generative Latent Flow Planning for World Models
arXiv:2608.24855v1 Announce Type: new Abstract: Latent world models are inherently strong encoders that transform image pixel to latent embedding, yet existing world models still rely on online traje...
Hugging Face Trending Papers
Aug 27
Decoupling Planning and Control for Instructable Agents
The paper introduces Instruct-to-Act, a system that decouples high‑level planning from low‑latency control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them autonomously at high frequency. Experiments across seven embodied environments, including multi‑agent settings, show that this approach outperforms both controller‑only and direct VLM action‑generation methods, maintains fast control, and allows swapping in different pretrained VLM planners without fine‑tuning.
arXiv AI
Aug 28
Decoupling Planning and Control for Instructable Agents
The paper introduces Instruct-to-Act, a system that decouples planning and control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them at high frequency, trained via relabeling rollouts with synthetic instructions and joint optimization of behavior cloning, reward, and world‑model objectives. Across seven embodied environments—including multi‑agent settings—this approach outperforms controller‑only and direct VLM action methods, maintains fast control, and allows swapping pretrained VLM planners without fine‑tuning, achieving competitive results with strong baselines on most tasks.
arXiv AI
6d ago
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
arXiv:2609.37250v1 Announce Type: cross Abstract: World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrai...