arXiv AI By Syed Nazmus Sakib, Abdul Monaf Chowdhury, Nafiul Haque, Shifat E Arman, Md Mehedi Hasan

Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?

Read the original on arXiv AI →

The paper investigates whether enhancing goal representations improves goal-conditioned reinforcement learning (GCRL) performance. By creating an exact temporal-distance goal representation in deterministic mazes and systematically degrading its geometric quality, the authors find that changes in goal representation have little effect on performance. In contrast, degrading the agent’s current state representation more than doubles failure rates, indicating that state representation is the critical bottleneck. The study further demonstrates that simple random Fourier positional encodings can significantly boost performance on challenging navigation tasks without additional map or objective modifications.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 2

Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning

The paper introduces Generalized Implicit Temporal Abstraction (GITA), a method for goal-conditioned reinforcement learning that conditions a single value function on multiple temporal abstraction levels (k). By aggregating advantage-weighted supervision across various k values, GITA preserves both long-range signal and local resolution without committing to a single k. Experiments on OGBench show that GITA outperforms existing offline GCRL baselines, improving average success rates by 25 percentage points over HIQL and 7 percentage points over OTA.

By Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira Jr., Marlos C. Machado
arXiv AI
Sep 30

HorizonFlow: Variable-Length Planning for Offline Goal-Conditioned RL

HorizonFlow is a hierarchical planner for offline goal-conditioned reinforcement learning that treats the planning horizon as an output rather than a fixed input. It uses a subgoal route planner and an action-prefix controller, both employing insertion-based generation and flow matching, to jointly generate continuous plan content and its length. The method leverages the partially generated plan to guide token insertion and to steer generation toward shorter plans, achieving superior performance on Maze2D, Multi2D, and OGBench benchmarks.

By JunHyeok Oh, Zian Jang, Byung-Jun Lee
arXiv AI
Oct 31

Escaping Path Mirages in Offline Goal-Conditioned Reinforcement Learning

The paper introduces Branch-aware Graph Planning (BGP), a method for offline goal-conditioned reinforcement learning that tackles the problem of path mirages in stochastic, long-horizon environments. BGP models stochastic branching by treating high-variability points as graph nodes and learns goal-conditioned edges with segmented learning to avoid unreliable branches and reduce unnecessary stochasticity. Experiments on diverse OGBench tasks demonstrate that BGP substantially outperforms existing state-of-the-art offline goal-conditioned RL approaches.

By Junhyeon Bae, Jaebak Hwang, Gwanwoo Choi, Minung Kim, Seungyul Han
arXiv Machine Learning
Oct 9

World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning

The paper introduces the World-Model Policy Arbiter (WMPA), a test‑time framework that selects among a portfolio of frozen goal‑conditioned policies by simulating each policy’s future trajectories in a learned world model and evaluating them with a shared value function. WMPA avoids frequent policy switching by committing to the chosen policy for a short interval and does not require retraining or task‑specific knowledge. On the OGBench benchmark, WMPA raises the macro‑average success rate from 44% to 58%, achieving significant improvements on 12 of 18 datasets, including large gains on cube‑double‑play and scene‑play.

By Junwei Quan, Evgenii Opryshko, Nicholas Rhinehart, Igor Gilitschenski