← Back to all news
arXiv AI By Jiayuan Wang, Tongyao Pang

DiffusionRVGE: Reinforcement Learning via Regression-Based Value Gradient Estimation

Read the original on arXiv AI →

DiffusionRVGE presents a reinforcement learning approach that improves diffusion models by estimating value gradients through regression of centered rewards on centered final denoised images, avoiding costly reward derivatives and backpropagation through the sampling trajectory. The method adapts rollout bandwidth and query weights based on reward continuity, enabling efficient training across multiple rewards. Experiments show that a single SD3.5‑Medium model trained with DiffusionRVGE matches or surpasses DiffusionNFT on most metrics while using roughly one‑fifth of the training compute, and achieves faster reward convergence in single‑reward settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Aug 26

On-Policy Self-Distillation in Diffusion Models

The paper introduces DiffusionOPSD, an on‑policy self‑distillation framework that transforms image‑level reinforcement learning rewards into explicit targets for intermediate denoising predictions in diffusion models. By generating trajectories with a frozen behavior policy and constructing bounded positive and negative targets around query states, the method trains a policy to fit these targets before updating the behavior policy via an exponential moving average. Experiments on SD 3.5‑M and Z‑Image‑Turbo show that DiffusionOPSD achieves the best held‑out scores in 19 of 20 reward‑matched settings, outperforms the strongest competitor by up to 44 % and cuts GPU‑hour usage by 40–63 % compared to DiffusionNFT.

By Wei Zhou, Xiongwei Zhu, Lingdong Kong, Bo Chen, Lei Zhang, Yongyuan Liang, Xiaoxia Hou, Ye Tian, Xian Sun, Yingshuo Wang, Linfeng Li, Shengqiong Wu, Leigang Qu, Feng Li, Wei Liu, Julian McAuley, Tat-Seng Chua
diffusionreinforcement-learningefficiencysafety
More like this →
arXiv Machine Learning
Oct 12

DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models

DiffusionART introduces a new reinforcement learning framework for diffusion models that aggregates prompt rewards into a shared direction, eliminating consistency forces inherent in prior methods. It employs a trust‑region‑masked second update for drift‑adaptive restoration, removing the need for EMA tuning and differentiable reward models. Experiments on image and video generation show that DiffusionART trains more efficiently and stably, achieving higher mean rewards with significantly fewer GPU hours compared to existing approaches.

By Zhenglin Wan, Jiayu Chen, Zhengfeiyang Zhang, Pengfei Zhou, Wangbo Zhao, Fangyuan Kong, Pengfei Wan, Kun Gai, Yang You
diffusionreinforcement-learning
More like this →
arXiv Machine Learning
Aug 5

Latent Reward Registers for Diffusion Preference Alignment

arXiv:2608. 03929v1 Announce Type: new Abstract: Aligning diffusion models with human preferences usually relies on a sparse terminal reward evaluated on the final generated samples, presenting a severe temporal credit-assignment challenge across the multi-step denoising process.

By Yuanshen Guan, Zipeng Feng, Zhiwei Xiong, Peiqin Sun
llmsdiffusionreinforcement-learningefficiencybenchmarkssafety
More like this →