arXiv Machine Learning

Amortizing intractable inference in diffusion models for vision, language, and control

The paper introduces a data‑free learning objective called relative trajectory balance for training diffusion models to sample from a posterior defined by a diffusion prior and an arbitrary black‑box constraint or likelihood. It proves asymptotic correctness of this objective and demonstrates its use across vision, language, and multimodal tasks, including classifier guidance, language infilling, and text‑to‑image generation. Additionally, the method is applied to continuous control with a score‑based behavior prior, achieving state‑of‑the‑art results in offline reinforcement learning.

arXiv Machine Learning

4d ago

Improved off-policy training of diffusion samplers

The paper investigates training diffusion models to sample from distributions defined by unnormalized densities or energy functions. It benchmarks various diffusion-structured inference techniques, including simulation-based variational methods and off-policy approaches such as continuous generative flow networks, highlighting their relative strengths and challenging some prior claims. Additionally, the authors introduce a new exploration strategy for off-policy methods that employs local search in the target space with a replay buffer, demonstrating improved sample quality across multiple target distributions.

By Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, Esmeralda S. Whitammer

diffusion benchmarks

arXiv AI

Jun 16

Sample from What You See: Visuomotor Policy Learning via Diffusion Bridge with Observation-Embedded Stochastic Differential Equation

arXiv:2512. 07212v3 Announce Type: replace Abstract: Imitation learning with diffusion models has advanced robotic control by capturing the multi-modal action distributions.

By Zhaoyang Liu, Mokai Pan, Zhongyi Wang, Kaizhen Zhu, Haotao Lu, Haipeng Zhang, Jingya Wang, Ye Shi

diffusion robotics multimodal benchmarks

arXiv Machine Learning

3d ago

Data-to-Energy Stochastic Dynamics

arXiv:2509.26364v3 Announce Type: replace Abstract: The Schr\"odinger bridge problem is concerned with finding a stochastic dynamical system bridging two marginal distributions that minimises a certa...

By Kirill Tamogashev, Esmeralda S. Whitammer

diffusion reinforcement-learning multimodal

arXiv Machine Learning

Jun 11

Reverse Flow Matching: A Unified Framework for Online Reinforcement Learning with Diffusion and Flow Policies

arXiv:2601. 08136v2 Announce Type: replace Abstract: Diffusion and flow policies are gaining prominence in online reinforcement learning (RL) due to their expressive power, yet training them efficiently remains a critical challenge.

By Zeyang Li, Sunbochen Tang, Navid Azizan

diffusion reinforcement-learning benchmarks

arXiv Machine Learning

Aug 5

Information-Geometric Forward Policy Training in GFlowNets

arXiv:2608. 03967v1 Announce Type: cross Abstract: Generative Flow Networks (GFlowNets) have emerged as a flexible framework for amortised inference over discrete and mixed discrete-continuous objects, requiring only an unnormalised target density specified through a reward.

By Yordan Raykov, Rodrigo Veiga

arXiv AI

Jun 11

Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies

arXiv:2602. 18291v2 Announce Type: replace Abstract: Online Multi-Agent Reinforcement Learning (MARL) is a prominent framework for efficient agent coordination.

By Zhuoran Li, Hai Zhong, Xun Wang, Qingxin Xia, Lihua Zhang, Longbo Huang

agents diffusion reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Aug 26

Scaling Reinforcement Learning for Diffusion Models via Velocity Matching

arXiv:2608.23664v1 Announce Type: cross Abstract: Reward fine-tuning is becoming an important tool for adapting diffusion models to human preferences and task-specific objectives, but existing method...

By Jaemoo Choi, Wei Guo, Yuchen Zhu, Arash Vahdat, Molei Tao, Julius Berner, Yongxin Chen

llms diffusion reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 20

Dichotomous Diffusion Policy Optimization

arXiv:2601. 00898v3 Announce Type: replace Abstract: Diffusion-based policies have gained growing popularity in solving a wide range of decision-making tasks due to their superior expressiveness and controllable generation during inference.

By Ruiming Liang, Yinan Zheng, Kexin Zheng, Tianyi Tan, Jianxiong Li, Liyuan Mao, Zhihao Wang, Guang Chen, Hangjun Ye, Jingjing Liu, Jinqiao Wang, Xianyuan Zhan

agents diffusion reinforcement-learning robotics multimodal benchmarks

arXiv AI

Jun 9

Latent Diffusion Policy: Shaping Latent Spaces for Diffusion-Based Robotic Manipulation

arXiv:2606. 08657v1 Announce Type: cross Abstract: Diffusion-based visuomotor policies operating directly in raw action spaces conflate scene comprehension with trajectory generation within a single denoising process.

By Zhexuan Zhou, Yichen Lai, Jinhao Zhang, Huizhe Li, Youmin Gong, Jie Mei

diffusion robotics

arXiv Machine Learning

Jun 19

DADP: Domain Adaptive Diffusion Policy

arXiv:2602. 04037v3 Announce Type: replace Abstract: Learning domain adaptive policies that can generalize to unseen transition dynamics, remains a fundamental challenge in learning-based control.

By Pengcheng Wang, Qinghang Liu, Haotian Lin, Yiheng Li, Guojian Zhan, Masayoshi Tomizuka, Yixiao Wang

diffusion robotics benchmarks

arXiv Machine Learning

Jun 16

DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising

arXiv:2606. 15359v1 Announce Type: new Abstract: Diffusion models have emerged as powerful tools for planning and control by learning multimodal distributions over actions and trajectories.

By Paolo Giaretta, Zeyang Li, Navid Azizan

diffusion multimodal benchmarks safety

arXiv Machine Learning

Jun 26

Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models

arXiv:2510. 09976v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models such as OpenVLA, Octo, and $\pi_0$ have shown strong generalization by leveraging large-scale demonstrations, yet their performance is still fundamentally constrained by the quality and coverage of supervised data.

By Mingyang Lyu, Yinqian Sun, Erliang Lin, Huangrui Li, Ruolin Chen, Feifei Zhao, Yi Zeng

diffusion reinforcement-learning fine-tuning multimodal benchmarks

Related stories