HorizonFlow is a hierarchical planner for offline goal-conditioned reinforcement learning that treats the planning horizon as an output rather than a fixed input. It uses a subgoal route planner and an action-prefix controller, both employing insertion-based generation and flow matching, to jointly generate continuous plan content and its length. The method leverages the partially generated plan to guide token insertion and to steer generation toward shorter plans, achieving superior performance on Maze2D, Multi2D, and OGBench benchmarks.
By JunHyeok Oh, Zian Jang, Byung-Jun Lee
Recent advances in generative planning have made trajectory inpainting a promising approach to offline goal-conditioned reinforcement learning. However, these methods typically specify the planning ho...
arXiv:2608.29061v1 Announce Type: new
Abstract: Offline goal-conditioned reinforcement learning (GCRL) aims to learn policies for reaching diverse goals entirely from fixed trajectory data. Long-hori...
By Soohyun Choi, Seonvin Cho, Songnam Hong
arXiv:2607. 20834v1 Announce Type: new Abstract: Offline goal-conditioned reinforcement learning (RL) holds the promise of learning general-purpose policies from static datasets.
By Ahad Jawaid
arXiv:2610.18677v1 Announce Type: cross
Abstract: Online decision-making faces a tension between efficient inference and long-horizon reasoning. Value-based methods rely on short-horizon bootstrappin...
By Haldun Balim, Yilun Du, Na Li
Offline goal-conditioned reinforcement learning (RL) holds the promise of learning general-purpose policies from static datasets. However, scaling these methods to long-horizon tasks remains a challenge due to the curse of horizon, where value estimation errors can compound through long chains of bootstrapped Bellman backups.
The paper introduces DCRL (Divide-and-Conquer RL), a method that recursively decomposes offline goal-conditioned reinforcement learning trajectories into a balanced binary tree. By training values from the leaves up to the root, DCRL avoids noisy max-based backups and reduces bootstrap depth from linear to logarithmic, thereby limiting error accumulation. Experiments on diverse goal-reaching tasks show that DCRL outperforms prior flat offline GCRL methods, achieving a higher average score on the most challenging long-horizon OGBench tasks.
By Hyeonseong Jeon, Youngwoon Lee
arXiv:2610.27280v1 Announce Type: cross
Abstract: Offline goal-conditioned reinforcement learning enables skill acquisition by reaching any goal from any state without dense rewards. Model-based meth...
By Yunzhe Hu, Dong Xu
arXiv:2609.38383v1 Announce Type: cross
Abstract: Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without po...
By Deqian Kong, Guangyan Sun, Sheng Cheng, Sirui Xie, Bo Pang, Jianwen Xie, Tony Geng, Caiwen Ding, Ying Nian Wu
arXiv:2607. 12547v1 Announce Type: cross Abstract: We investigate whether temporal hierarchy can improve LeWorldModel on long-horizon goal-conditioned control.
By Niccol\`o Caselli, Salvatore Lo Sardo, Francesco Massafra, Ippokratis Pantelidis, Samuele Punzo, Sathya Kamesh Bhethanabhotla
arXiv:2601. 22496v2 Announce Type: replace-cross Abstract: In offline goal-conditioned reinforcement learning (GCRL), hierarchical approaches decompose long-horizon tasks into high-level subgoal prediction and low-level action execution.
By Jinu Hyeon, Woobin Park, Hongjoon Ahn, Taesup Moon
The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.
By Delin Zhao, Zhengrong Yue, Shaobin Zhuang, Junlin He, Xiaoyu Chen, Zikang Wang, Yuxin Liu, Limin Wang, Yali Wang