← Back to all news
arXiv Computer Vision By Junlong Wu, Jiuzhou Lin, Jia Sun, Boheng Zhang, Huaiqing Wang, Dewen Fan, Houde Liu, Qianqian Gan, Fan Yang, Tingting Gao

Step Back to Move Forward: Reflection-Aware Preference Optimization for Visual Generation

Read the original on arXiv Computer Vision →

The paper introduces Reflection-Aware GRPO (RA‑GRPO), a reinforcement‑learning framework that aligns diffusion generative models with human preferences. It uses Diffusion Reflection to correct intermediate sampling paths by reversing the diffusion process, and Counterfactual Path Synthesis to embed these corrected trajectories into the policy, avoiding extra inference cost. Experiments on text‑to‑image and text‑to‑video models show RA‑GRPO outperforms existing methods, reducing reward hacking and improving generalization while remaining architecture‑agnostic.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Sep 7

Learning to Credit the Right Steps: Objective-aware Process Optimization for Visual Generation

The paper introduces Objective-aware Trajectory Credit Assignment (OTCA), a framework that refines reinforcement learning for diffusion-based visual generation. OTCA decomposes credit across denoising steps and allocates multiple reward signals adaptively, addressing the coarse, uniform reward assignment of existing GRPO pipelines. Experiments demonstrate that OTCA consistently enhances image and video generation quality across various metrics.

By Rui Li, Ke Hao, Yuanzhi Liang, Haibin Huang, Chi Zhang, Yun Gu, Xuelong Li
diffusionreinforcement-learningsafety
More like this →
arXiv Machine Learning
Sep 7

Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models

The paper introduces Diffusion LAIR, a listwise preference optimization technique that leverages continuous reward scores instead of binary pairwise comparisons to align text‑to‑image diffusion models. LAIR transforms reward scores into centered advantage weights and optimizes an advantage‑weighted regression objective on an implicit reward defined by denoising‑loss improvement over a reference model, with a quadratic penalty to regulate reward magnitude. Experiments demonstrate that Diffusion LAIR surpasses strong baseline methods on SD1.5 and SDXL across generation, compositional, and editing tasks.

By Austin Wang, Jiaqi Han, Stefano Ermon, Yisong Yue
diffusionreinforcement-learningbenchmarkssafety
More like this →
arXiv AI
Jul 1

Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models

arXiv:2603. 12893v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a standard technique for post-training diffusion-based image synthesis models, as it enables learning from reward signals to explicitly improve desirable aspects such as image quality and prompt alignment.

By David McAllister, Miika Aittala, Tero Karras, Janne Hellsten, Angjoo Kanazawa, Timo Aila, Samuli Laine
llmsdiffusionreinforcement-learningsafety
More like this →