arXiv:2610.01723v1 Announce Type: new
Abstract: Text-to-image diffusion models have achieved remarkable progress in image synthesis, yet can exhibit memorization by closely reproducing individual tra...
By Hyungjun Joo, Sehwan Kim, Hyeonggeun Han, Sangwoo Hong, Jungwoo Lee
The paper introduces Reflection-Aware GRPO (RA‑GRPO), a reinforcement‑learning framework that aligns diffusion generative models with human preferences. It uses Diffusion Reflection to correct intermediate sampling paths by reversing the diffusion process, and Counterfactual Path Synthesis to embed these corrected trajectories into the policy, avoiding extra inference cost. Experiments on text‑to‑image and text‑to‑video models show RA‑GRPO outperforms existing methods, reducing reward hacking and improving generalization while remaining architecture‑agnostic.
By Junlong Wu, Jiuzhou Lin, Jia Sun, Boheng Zhang, Huaiqing Wang, Dewen Fan, Houde Liu, Qianqian Gan, Fan Yang, Tingting Gao
arXiv:2603. 12893v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a standard technique for post-training diffusion-based image synthesis models, as it enables learning from reward signals to explicitly improve desirable aspects such as image quality and prompt alignment.
By David McAllister, Miika Aittala, Tero Karras, Janne Hellsten, Angjoo Kanazawa, Timo Aila, Samuli Laine
Efficient Adjoint Matching (EAM) is a new method for reward fine‑tuning of pretrained diffusion models that reduces computational cost by reformulating the stochastic optimal control problem with a linear base drift and a modified terminal cost. This change eliminates the need for extensive stochastic simulation of full generative trajectories and backward ODE simulation of the adjoint state, allowing training‑time sampling with a few‑step deterministic ODE solver and a closed‑form adjoint solution. On standard text‑to‑image reward fine‑tuning benchmarks, EAM converges up to four times faster than the previous Adjoint Matching approach while matching or surpassing it on metrics such as PickScore, ImageReward, HPSv2.1, CLIPScore, and Aesthetics.
By Jeongwoo Shin, Dongsoo Shin, Yuchen Zhu, Wei Guo, Yongxin Chen, Joonseok Lee, Jaewoong Choi, Jaemoo Choi
arXiv:2608. 14038v1 Announce Type: new Abstract: Modern text-to-image diffusion models rely on classifier-free guidance (CFG) to achieve high image fidelity and text alignment.
By Ashwini Pokle, Alexandre Galashov, Arnaud Doucet, Mauricio Delbracio, Valentin De Bortoli
arXiv:2608.23664v1 Announce Type: cross
Abstract: Reward fine-tuning is becoming an important tool for adapting diffusion models to human preferences and task-specific objectives, but existing method...
By Jaemoo Choi, Wei Guo, Yuchen Zhu, Arash Vahdat, Molei Tao, Julius Berner, Yongxin Chen
The paper introduces a data‑free learning objective called relative trajectory balance for training diffusion models to sample from a posterior defined by a diffusion prior and an arbitrary black‑box constraint or likelihood. It proves asymptotic correctness of this objective and demonstrates its use across vision, language, and multimodal tasks, including classifier guidance, language infilling, and text‑to‑image generation. Additionally, the method is applied to continuous control with a score‑based behavior prior, achieving state‑of‑the‑art results in offline reinforcement learning.
By Siddarth Venkatraman, Moksh Jain, Luca Scimeca, Minsu Kim, Marcin Sendera, Mohsin Hasan, Luke Rowe, Sarthak Mittal, Pablo Lemos, Emmanuel Bengio, Alexandre Adam, Jarrid Rector-Brooks, Yoshua Bengio, Glen Berseth, Esmeralda S. Whitammer
arXiv:2603. 00133v2 Announce Type: replace-cross Abstract: Generative models have been shown to "memorize" certain training data, leading to verbatim or near-verbatim generating images, which may cause privacy concerns or copyright infringement.
By Kairan Zhao, Eleni Triantafillou, Peter Triantafillou
Text-to-image diffusion models expose many inference-time sampling parameters, including prompts, negative prompts, classifier-free guidance scales, and noise schedules. These parameters are typically manually chosen once and then held fixed across prompts and denoising timesteps, even though different prompts and stages of generation can benefit from different parameter values.
Routing Policy Optimization (RPO) introduces stochastic exploration within a diffusion visual generator by treating attention routing as a stochastic action. Instead of perturbing the entire reverse denoising trajectory, RPO samples key attention blocks at selected denoising levels, perturbs the transition velocity, and evaluates the perturbation with a score‑function objective. A two‑rollout estimator provides an advantage signal, and after training the model restores dense attention, yielding higher generation quality and nearly four times the training efficiency compared to Flow‑GRPO‑style methods.
The paper introduces Learned End-to-End Guidance Schedules (LEEGS) for diffusion models, which train a time‑dependent guidance schedule to balance data quality and requirement satisfaction while reducing sampling steps. LEEGS minimizes the guidance function over a small set of examples using stochastic gradient descent and employs a gradient approximation to cut training time by a factor of four. Experiments on tasks such as image inpainting, noisy image inverse problems, face‑ID‑guided generation, and PDE problems show that LEEGS outperforms baselines at the same computational budget or matches constant guidance with only 10% of the steps.
By Aneesh Barthakur, Mathias Niepert, Luiz F. O. Chamon
arXiv:2607. 23488v1 Announce Type: new Abstract: Text-to-image diffusion models expose many inference-time sampling parameters, including prompts, negative prompts, classifier-free guidance scales, and noise schedules.
By Arisrei Lim, Yossi Gandelsman