← Back to all news
arXiv Machine Learning By Shashank Gupta, Pilhwa Lee

Density-Guided Bootstrap Boundaries for Latent World-Model Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces density‑guided bootstrap boundaries for latent world‑model reinforcement learning, a trajectory‑dependent method that adjusts the depth of imagined returns based on a score‑informed relative density proxy. Two gating mechanisms—hard and soft—are proposed to either halt return expansion at a discrete state or smoothly shift recursion toward critic bootstrapping, preserving boundary value estimates and applying survival weights to actor and critic losses. Implemented within Dreamer V3, the authors prove that the gated Bellman operator is a strict contraction and that early bootstrapping reduces conditional target variance, demonstrating improved sample efficiency and late‑training performance on continuous‑control tasks and Atari Pong compared to fixed‑horizon baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Statistics ML
Oct 1

Learning to Plan from Random Exploration

arXiv:2609.38383v1 Announce Type: cross Abstract: Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without po...

By Deqian Kong, Guangyan Sun, Sheng Cheng, Sirui Xie, Bo Pang, Jianwen Xie, Tony Geng, Caiwen Ding, Ying Nian Wu
ragagentsdiffusionrobotics
More like this →
arXiv AI
Sep 30

Beyond a single latent space: a dual-latent world model for long-horizon planning

The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.

By Delin Zhao, Zhengrong Yue, Shaobin Zhuang, Junlin He, Xiaoyu Chen, Zikang Wang, Yuxin Liu, Limin Wang, Yali Wang
diffusion
More like this →
arXiv Machine Learning
Oct 8

An Informational Curse of Horizon in Goal-Conditioned Policy Learning

The paper identifies an informational curse of horizon in goal-conditioned policy learning, showing that longer goal relabeling horizons during training can hurt policy generalization and performance. Controlled experiments with oracle planners separate training horizons from test horizons, revealing that behavioral cloning policies degrade with longer horizons while reinforcement learning mitigates this effect. The authors attribute this to reduced conditional mutual information between actions and hindsight-relabeled goals, and demonstrate that distilling short-horizon policy Jacobians into long-horizon policies improves performance, especially in combinatorial manipulation tasks.

By John L. Zhou, Yuxuan Dong, Jonathan C. Kao
reinforcement-learningroboticssafety
More like this →