arXiv Machine Learning By Alexis Jacq, Guillaume Couairon, Valentin De Bortoli, Quentin Berthet, Arnaud Doucet, Romuald Elie

Diffusion Fine-tuning with Rewarded Moment Matching Distillation

Read the original on arXiv Machine Learning →

arXiv:2606. 30414v1 Announce Type: new Abstract: Distillation and Reinforcement Learning (RL) fine-tuning are the primary pillars of diffusion post-training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computer Vision
Aug 26

On-Policy Self-Distillation in Diffusion Models

The paper introduces DiffusionOPSD, an on‑policy self‑distillation framework that transforms image‑level reinforcement learning rewards into explicit targets for intermediate denoising predictions in diffusion models. By generating trajectories with a frozen behavior policy and constructing bounded positive and negative targets around query states, the method trains a policy to fit these targets before updating the behavior policy via an exponential moving average. Experiments on SD 3.5‑M and Z‑Image‑Turbo show that DiffusionOPSD achieves the best held‑out scores in 19 of 20 reward‑matched settings, outperforms the strongest competitor by up to 44 % and cuts GPU‑hour usage by 40–63 % compared to DiffusionNFT.

By Wei Zhou, Xiongwei Zhu, Lingdong Kong, Bo Chen, Lei Zhang, Yongyuan Liang, Xiaoxia Hou, Ye Tian, Xian Sun, Yingshuo Wang, Linfeng Li, Shengqiong Wu, Leigang Qu, Feng Li, Wei Liu, Julian McAuley, Tat-Seng Chua
arXiv AI
2d ago

CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models

CAST introduces a reinforcement‑learning fine‑tuning framework for diffusion models that addresses three key limitations: it automatically selects the denoising window based on each model’s trajectory, decomposes prompts into verifiable semantic atoms via Causal Scene Graphs, and applies atom‑level rewards spatially weighted in the policy objective. The method is applied to FLUX.2‑dev and Qwen‑Image‑2512, yielding up to 3.07× improvement on the hardest GenEval 2 prompts compared with Flow‑GRPO while also enhancing overall generation quality.

By Shu Yu, Chaochao Lu