arXiv:2607. 14522v1 Announce Type: new Abstract: We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynamics are modeled as a controlled continuous-time Markov chain (CTMC).
By Zikun Zhang, Jiayuan Sheng, David D. Yao, Wenpin Tang
arXiv:2607. 07693v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) has emerged as a powerful paradigm for aligning generative models with human preferences.
By Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay
arXiv:2601. 00898v3 Announce Type: replace Abstract: Diffusion-based policies have gained growing popularity in solving a wide range of decision-making tasks due to their superior expressiveness and controllable generation during inference.
By Ruiming Liang, Yinan Zheng, Kexin Zheng, Tianyi Tan, Jianxiong Li, Liyuan Mao, Zhihao Wang, Guang Chen, Hangjun Ye, Jingjing Liu, Jinqiao Wang, Xianyuan Zhan
arXiv:2606. 15048v1 Announce Type: new Abstract: Diffusion models are typically trained with objectives that focus on local denoising targets at individual time steps (or adjacent pairs), which do not enforce consistency between predictions along the denoising trajectory.
By Qizhen Ying, Yangchen Pan, Victor Adrian Prisacariu, Junfeng Wen
The paper reviews how reinforcement learning (RL) can refine diffusion models (DMs) after pretraining, addressing limitations in reasoning, exploration, and alignment. It surveys RL-driven DMs across language, multimodal, robotic, and scientific domains, focusing on trajectory optimization and reward engineering. Experiments highlight the benefits of various RL post‑training strategies and identify key bottlenecks—reward fragility, exploration‑exploitation trade‑offs, and training instability—while proposing a roadmap for future research and a public repository for updates.
By Zhiyuan Ma, Jiabao Wei, Tianyu Wang, Hanqing Wang, Jiadong Pan, Haowen Xu, Liangliang Zhao, Ruixun Liu, Yuhan Wang, Yuwei Zhang, Liang Feng, Junhao Xiao, Kaiyan Zhang, Xuekai Zhu, Daizong Liu, Biqing Qi, Yihao Liu, Ganqu Cui, Ning Ding, Linfeng Zhang, Jianjun Li, Kun He, Xiang Bai, Bowen Zhou
arXiv:2608. 14430v1 Announce Type: new Abstract: Reinforcement learning (RL) post-training provides a direct way to align diffusion models with human preferences and task-specific rewards.
By Yixian Xu, Yuanrui Zhang, Shengjie Luo, Liwei Wang, Di He
arXiv:2605.08733v2 Announce Type: replace
Abstract: Expressive generative policies such as diffusion and flow models are appealing for maximum-entropy (MaxEnt) online reinforcement learning because o...
By Ke He, Le He, Shunpu Tang, Yafei Wang, Lisheng Fan
The paper introduces Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints (FMER), a reinforcement learning framework that uses ODE-based flow matching to overcome limitations of SDE-based diffusion policies. FMER derives an analytic entropy objective to correct density distortion from a tanh transformation, establishes that minimizing an advantage-weighted conditional flow matching loss serves as a simulation-free surrogate for policy mirror descent, and dynamically tunes the mirror descent temperature to enforce a robust trust region. Experiments show FMER outperforms baselines on the sparse-reward FrankaKitchen task while remaining competitive on dense-reward MuJoCo benchmarks.
By Ting Gao, Stavros Orfanoudakis, Nan Lin, Winnie Daamen, Serge Hoogendoorn, Elvin Isufi
arXiv:2606. 10825v1 Announce Type: new Abstract: Diffusion policies (DPs) have emerged as expressive policy representations for robot learning, often used with imitation learning methods such as behavioral cloning (BC).
By Zakariae El Asri, Philippe Gratias-Quiquandon, Nicolas Thome, Olivier Sigaud
Efficient Diffusion Policy Fine Tuning with Latent Noise Representation Bridging introduces the Latent Noise Linear Markov Decision Process (LNL‑MDP) to address the noise aliasing problem in fine‑tuning pre‑trained diffusion and flow policies for robotics. By deriving a Bridge Equation that aligns noise and action spaces through a shared representation, the authors provide a framework that falls under FineTuneRL theory with near‑optimal guarantees. The resulting LARBRID algorithm learns this representation to optimize the initial noise distribution, achieving 2–3× performance gains on a variety of robotic locomotion and manipulation tasks.
By Yuanchu Liang, Yue Yang, An-Chi He, Hanna Kurniawati
arXiv:2606. 08602v1 Announce Type: cross Abstract: We present an online reinforcement learning (RL) algorithm for fine-tuning flow-matching policies in continuous-control problems.
By Boshu Lei, Kostas Daniilidis, Antonio Loquercio
arXiv:2602. 18291v2 Announce Type: replace Abstract: Online Multi-Agent Reinforcement Learning (MARL) is a prominent framework for efficient agent coordination.
By Zhuoran Li, Hai Zhong, Xun Wang, Qingxin Xia, Lihua Zhang, Longbo Huang