arXiv:2607. 20834v1 Announce Type: new Abstract: Offline goal-conditioned reinforcement learning (RL) holds the promise of learning general-purpose policies from static datasets.
Offline goal-conditioned reinforcement learning (RL) holds the promise of learning general-purpose policies from static datasets. However, scaling these methods to long-horizon tasks remains a challenge due to the curse of horizon, where value estimation errors can compound through long chains of bootstrapped Bellman backups.
The paper introduces DCRL (Divide-and-Conquer RL), a method that recursively decomposes offline goal-conditioned reinforcement learning trajectories into a balanced binary tree. By training values from the leaves up to the root, DCRL avoids noisy max-based backups and reduces bootstrap depth from linear to logarithmic, thereby limiting error accumulation. Experiments on diverse goal-reaching tasks show that DCRL outperforms prior flat offline GCRL methods, achieving a higher average score on the most challenging long-horizon OGBench tasks.
arXiv:2603. 01891v2 Announce Type: replace Abstract: Action chunking improves exploration and accelerates value propagation in long-horizon reinforcement learning, but naively applying off-policy methods to the temporally extended action space at reduced decision frequency offsets these gains, leading to poor sample efficiency.
HorizonFlow is a hierarchical planner for offline goal-conditioned reinforcement learning that treats the planning horizon as an output rather than a fixed input. It uses a subgoal route planner and an action-prefix controller, both employing insertion-based generation and flow matching, to jointly generate continuous plan content and its length. The method leverages the partially generated plan to guide token insertion and to steer generation toward shorter plans, achieving superior performance on Maze2D, Multi2D, and OGBench benchmarks.
The paper introduces Q-Shaped Options (QSO), a hierarchical reinforcement learning framework that learns action abstractions between consecutive levels using shared encoders shaped by Q functions. QSO employs distinct state-value functions, Q functions, and policies at each hierarchy level, encouraging the low-level Q function to preserve distinctions needed for optimal control while the high-level Q function discards unnecessary distinctions. Experiments on offline goal-conditioned locomotion and manipulation tasks show that QSO learns semantically meaningful option spaces and outperforms baselines, achieving non-zero performance where other algorithms fail.
Grounded Transitive RL (GTRL) is an offline goal‑conditioned reinforcement learning algorithm that improves upon divide‑and‑conquer value learning by grounding updates with a one‑step temporal‑difference (TD) target. By adding this TD target rather than replacing it, GTRL ensures every state‑goal pair receives an update and corrects bias from hindsight relabeling through reweighting based on reachability. The method was evaluated on nineteen OGBench tasks across stochastic, deterministic, and stitching environments, achieving the highest average success rate among compared approaches.
The paper introduces Generalized Implicit Temporal Abstraction (GITA), a method for goal-conditioned reinforcement learning that conditions a single value function on multiple temporal abstraction levels (k). By aggregating advantage-weighted supervision across various k values, GITA preserves both long-range signal and local resolution without committing to a single k. Experiments on OGBench show that GITA outperforms existing offline GCRL baselines, improving average success rates by 25 percentage points over HIQL and 7 percentage points over OTA.
Recent advances in generative planning have made trajectory inpainting a promising approach to offline goal-conditioned reinforcement learning. However, these methods typically specify the planning ho...
arXiv:2512. 09706v2 Announce Type: replace Abstract: The paradigm of agentic AI is shifting from engineered complex workflows to post-training native models.
arXiv:2606. 04188v1 Announce Type: cross Abstract: Offline goal-conditioned reinforcement learning requires both long-horizon reachability estimates and local action comparisons.
arXiv:2601. 22496v2 Announce Type: replace-cross Abstract: In offline goal-conditioned reinforcement learning (GCRL), hierarchical approaches decompose long-horizon tasks into high-level subgoal prediction and low-level action execution.