arXiv Machine Learning

Bayesian Flow Networks for Offline Trajectory Planning

The paper introduces Bayesian Flow Networks for Offline Trajectory Planning (BFN-RL), a generative modeling framework that unifies discrete and continuous trajectory synthesis for offline reinforcement learning. Unlike prior diffusion models that rely on Gaussian noise, BFN-RL iteratively updates distribution parameters, enabling a categorical planner to produce future state sequences and an inverse-dynamics model to translate these states into actions. Experiments demonstrate that BFN-RL effectively generates trajectories in both discrete planning and continuous control tasks, highlighting its versatility across data modalities.

arXiv AI

4d ago

HorizonFlow: Variable-Length Planning for Offline Goal-Conditioned RL

HorizonFlow is a hierarchical planner for offline goal-conditioned reinforcement learning that treats the planning horizon as an output rather than a fixed input. It uses a subgoal route planner and an action-prefix controller, both employing insertion-based generation and flow matching, to jointly generate continuous plan content and its length. The method leverages the partially generated plan to guide token insertion and to steer generation toward shorter plans, achieving superior performance on Maze2D, Multi2D, and OGBench benchmarks.

By JunHyeok Oh, Zian Jang, Byung-Jun Lee

diffusion reinforcement-learning robotics benchmarks

Hugging Face Trending Papers

4d ago

HorizonFlow: Variable-Length Planning for Offline Goal-Conditioned RL

Recent advances in generative planning have made trajectory inpainting a promising approach to offline goal-conditioned reinforcement learning. However, these methods typically specify the planning ho...

diffusion reinforcement-learning robotics benchmarks

arXiv Machine Learning

2d ago

In-Distribution Imagination for Model-Based Offline Reinforcement Learning

arXiv:2609.38673v1 Announce Type: new Abstract: Model-based offline reinforcement learning (MBORL) improves sample efficiency through model-generated trajectories. However, accumulative model error c...

By Mintae Kim, Koushil Sreenath

reinforcement-learning

arXiv AI

Jun 29

RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidance

arXiv:2606. 27766v1 Announce Type: cross Abstract: Offline reinforcement learning enables policy learning from fixed datasets without additional environment interaction, making it appealing for safety-critical applications where online exploration is costly or unsafe.

By Shiqiang Gong

diffusion reinforcement-learning robotics multimodal benchmarks safety

arXiv Machine Learning

Jun 16

Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernels

arXiv:2606. 16073v1 Announce Type: new Abstract: Sampling from complex, unnormalized probability densities is a fundamental challenge in Bayesian inference and probabilistic modeling.

By Kirill Korolev, Nikita Morozov, Stepan Pavlenko, Esmeralda S. Whitammer, Sergey Samsonov

benchmarks

arXiv Machine Learning

Jun 26

Normalizing Flows are Capable Models for Continuous Control

arXiv:2505. 23527v4 Announce Type: replace Abstract: Modern reinforcement learning (RL) algorithms have found success by using powerful probabilistic models, such as transformers, energy-based models, and diffusion/flow-based models.

By Raj Ghugare, Benjamin Eysenbach

llms diffusion reinforcement-learning

arXiv AI

Jun 19

VOiLA: Vectorized Online Planning with Learned Diffusion Model for POMDP Agents

arXiv:2606. 19729v1 Announce Type: cross Abstract: Planning under uncertainty is an essential capability for autonomous robots.

By Marcus Hoerger, Rishikesh Joshi, Rahul Shome, Ian Manchester, Hanna Kurniawati

agents diffusion reinforcement-learning robotics efficiency benchmarks

arXiv AI

Jul 17

VOiLA: Vectorized Online Planning with Learned Diffusion Models for POMDP Agents

arXiv:2606. 19729v2 Announce Type: replace-cross Abstract: Planning under uncertainty is an essential capability for autonomous robots.

By Marcus Hoerger, Rishikesh Joshi, Rahul Shome, Ian Manchester, Hanna Kurniawati

agents diffusion reinforcement-learning robotics efficiency benchmarks

arXiv AI

Jul 13

Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning

arXiv:2607. 09336v1 Announce Type: cross Abstract: Diffusion-based trajectory planners have shown strong performance in offline reinforcement learning, but their iterative denoising process often incurs high inference cost.

By Guanquan Wang, Yoshimasa Tsuruoka

diffusion reinforcement-learning robotics efficiency benchmarks

arXiv Machine Learning

Jun 8

GenPO++: Generative Policy Optimization with Jacobian-free Likelihood Ratios

arXiv:2606. 06967v1 Announce Type: new Abstract: Generative policies provide expressive and multimodal action distributions, making them attractive for reinforcement learning (RL) in complex continuous-control tasks.

By Ke Hu, Shutong Ding, Panxin Tao, Jingya Wang, Ye Shi

reinforcement-learning robotics fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Aug 31

Trajectory balance: Improved credit assignment in GFlowNets

The paper introduces a new learning objective called trajectory balance for Generative Flow Networks (GFlowNets), aiming to improve credit assignment across long action sequences. It demonstrates that minimizing this objective yields a policy that samples exactly from the target distribution. Experiments on four domains show that trajectory balance enhances convergence, sample diversity, and robustness to long sequences and large action spaces.

By Esmeralda S. Whitammer, Moksh Jain, Emmanuel Bengio, Chen Sun, Yoshua Bengio

diffusion

arXiv Computer Vision

Aug 21

RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation

arXiv:2608. 20208v1 Announce Type: new Abstract: Offline reinforcement learning improves robotic policies using previously collected data without further environment interaction.

By Shaoxuan Wang, Guangting Zheng, Rui Huang, Zhipeng Tang, Sha Zhang, Jiajun Deng, Yanyong Zhang

diffusion reinforcement-learning robotics efficiency benchmarks

Related stories