arXiv Machine Learning July 17, 2026 By Zikun Zhang, Jiayuan Sheng, David D. Yao, Wenpin Tang

A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models

Read the original on arXiv Machine Learning →

arXiv:2607. 14522v1 Announce Type: new Abstract: We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynamics are modeled as a controlled continuous-time Markov chain (CTMC).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

2d ago

Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning

The paper introduces Diffusion-Augmented Markov Decision Processes (DA‑MDPs), a framework that extends Maximum Entropy Reinforcement Learning to diffusion-based policies. DA‑MDPs treat each reverse‑diffusion step as an RL decision, deriving a tractable reverse‑KL bound that decomposes across denoising transitions and yields diffusion‑augmented soft rewards, value functions, and policy objectives. The authors implement this framework with PPO, REPPO, and a maximum‑entropy WPO variant, showing improved continuous‑control performance, higher success rates on manipulation tasks, and memory‑efficient training with action chunking.

By Sebastian Sanokowski, Kaustubh Patil, Majid Khadiv

rag diffusion reinforcement-learning robotics

arXiv Machine Learning

Aug 26

Scaling Reinforcement Learning for Diffusion Models via Velocity Matching

arXiv:2608.23664v1 Announce Type: cross Abstract: Reward fine-tuning is becoming an important tool for adapting diffusion models to human preferences and task-specific objectives, but existing method...

By Jaemoo Choi, Wei Guo, Yuchen Zhu, Arash Vahdat, Molei Tao, Julius Berner, Yongxin Chen

llms diffusion reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 20

Dichotomous Diffusion Policy Optimization

arXiv:2601. 00898v3 Announce Type: replace Abstract: Diffusion-based policies have gained growing popularity in solving a wide range of decision-making tasks due to their superior expressiveness and controllable generation during inference.

By Ruiming Liang, Yinan Zheng, Kexin Zheng, Tianyi Tan, Jianxiong Li, Liyuan Mao, Zhihao Wang, Guang Chen, Hangjun Ye, Jingjing Liu, Jinqiao Wang, Xianyuan Zhan

agents diffusion reinforcement-learning robotics multimodal benchmarks

arXiv AI

Jul 9

Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF

arXiv:2607. 07693v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) has emerged as a powerful paradigm for aligning generative models with human preferences.

By Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay

diffusion reinforcement-learning

arXiv AI

Jun 10

Model-Based Reinforcement Learning in Discrete-Action Non-Markovian Reward Decision Processes

arXiv:2512. 14617v2 Announce Type: replace-cross Abstract: Many practical decision-making problems involve tasks whose success depends on the entire system history, rather than on achieving a state with desired properties.

By Alessandro Trapasso, Luca Iocchi, Fabio Patrizi

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 16

Temporal Difference Learning for Diffusion Models

arXiv:2606. 15048v1 Announce Type: new Abstract: Diffusion models are typically trained with objectives that focus on local denoising targets at individual time steps (or adjacent pairs), which do not enforce consistency between predictions along the denoising trajectory.

By Qizhen Ying, Yangchen Pan, Victor Adrian Prisacariu, Junfeng Wen

diffusion reinforcement-learning benchmarks

One email a morning, machine-written

Related stories

Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning

Scaling Reinforcement Learning for Diffusion Models via Velocity Matching

Dichotomous Diffusion Policy Optimization

Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF

Model-Based Reinforcement Learning in Discrete-Action Non-Markovian Reward Decision Processes

Temporal Difference Learning for Diffusion Models