arXiv AI By Sathwik Karnik, Juyeop Kim, Sanmi Koyejo, Jong-Seok Lee, Somil Bansal

Steering Away from Memorization: Reachability-Constrained Reinforcement Learning for Text-to-Image Diffusion

Read the original on arXiv AI →

The paper introduces Reachability-Aware Diffusion Steering (RADS), an offline-trained, sampling-time control framework designed to reduce memorization in text-to-image diffusion models while preserving image quality and prompt alignment. RADS treats the diffusion denoising process as a dynamical system and learns a policy-conditioned reachability value function that predicts future memorization risk along the denoising trajectory. By formulating a constrained reinforcement learning problem, the policy steers the trajectory away from memorization through perturbations in the caption embedding space, achieving a better trade‑off between generation diversity, quality, and alignment than existing inference‑time baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Sep 7

Step Back to Move Forward: Reflection-Aware Preference Optimization for Visual Generation

The paper introduces Reflection-Aware GRPO (RA‑GRPO), a reinforcement‑learning framework that aligns diffusion generative models with human preferences. It uses Diffusion Reflection to correct intermediate sampling paths by reversing the diffusion process, and Counterfactual Path Synthesis to embed these corrected trajectories into the policy, avoiding extra inference cost. Experiments on text‑to‑image and text‑to‑video models show RA‑GRPO outperforms existing methods, reducing reward hacking and improving generalization while remaining architecture‑agnostic.

By Junlong Wu, Jiuzhou Lin, Jia Sun, Boheng Zhang, Huaiqing Wang, Dewen Fan, Houde Liu, Qianqian Gan, Fan Yang, Tingting Gao
arXiv AI
Jul 1

Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models

arXiv:2603. 12893v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a standard technique for post-training diffusion-based image synthesis models, as it enables learning from reward signals to explicitly improve desirable aspects such as image quality and prompt alignment.

By David McAllister, Miika Aittala, Tero Karras, Janne Hellsten, Angjoo Kanazawa, Timo Aila, Samuli Laine
arXiv Machine Learning
Oct 23

Efficient Adjoint Matching for Fine-tuning Diffusion Models

Efficient Adjoint Matching (EAM) is a new method for reward fine‑tuning of pretrained diffusion models that reduces computational cost by reformulating the stochastic optimal control problem with a linear base drift and a modified terminal cost. This change eliminates the need for extensive stochastic simulation of full generative trajectories and backward ODE simulation of the adjoint state, allowing training‑time sampling with a few‑step deterministic ODE solver and a closed‑form adjoint solution. On standard text‑to‑image reward fine‑tuning benchmarks, EAM converges up to four times faster than the previous Adjoint Matching approach while matching or surpassing it on metrics such as PickScore, ImageReward, HPSv2.1, CLIPScore, and Aesthetics.

By Jeongwoo Shin, Dongsoo Shin, Yuchen Zhu, Wei Guo, Yongxin Chen, Joonseok Lee, Jaewoong Choi, Jaemoo Choi