arXiv Machine Learning
Aug 27

Bayesian Flow Networks for Offline Trajectory Planning

The paper introduces Bayesian Flow Networks for Offline Trajectory Planning (BFN-RL), a generative modeling framework that unifies discrete and continuous trajectory synthesis for offline reinforcement learning. Unlike prior diffusion models that rely on Gaussian noise, BFN-RL iteratively updates distribution parameters, enabling a categorical planner to produce future state sequences and an inverse-dynamics model to translate these states into actions. Experiments demonstrate that BFN-RL effectively generates trajectories in both discrete planning and continuous control tasks, highlighting its versatility across data modalities.

By Ludvig Killingberg, Helge Langseth
arXiv AI
Sep 30

HorizonFlow: Variable-Length Planning for Offline Goal-Conditioned RL

HorizonFlow is a hierarchical planner for offline goal-conditioned reinforcement learning that treats the planning horizon as an output rather than a fixed input. It uses a subgoal route planner and an action-prefix controller, both employing insertion-based generation and flow matching, to jointly generate continuous plan content and its length. The method leverages the partially generated plan to guide token insertion and to steer generation toward shorter plans, achieving superior performance on Maze2D, Multi2D, and OGBench benchmarks.

By JunHyeok Oh, Zian Jang, Byung-Jun Lee
arXiv AI
Jun 2

Improving Diffusion Planners by Self-Supervised Action Gating with Energies

arXiv:2603. 02650v2 Announce Type: replace-cross Abstract: Diffusion planners are a strong approach for offline reinforcement learning, but they can fail when value-guided selection favours trajectories that score well yet are locally inconsistent with the environment dynamics, resulting in brittle execution.

By Yuan Lu, Dongqi Han, Yansen Wang, Dongsheng Li