The paper presents a post‑training approach for text‑to‑image models that combines a preference reward, trained on large human preference data, with rubric‑based rewards that assess prompt faithfulness and other desirable traits. The authors show that a simple reward composition strategy outperforms a naive weighted average, leading to significant Elo gains on the Arena leaderboard for models like Flux2dev and Ideogram‑4. They also release Arena‑T2I‑Training, a 1K subset of data to aid reproducible research in post‑training.
By Yuanhao Ban, I-Hung Hsu, Anastasios Angelopoulos, Wei-Lin Chiang, Ion Stoica, Cho-Jui Hsieh
The paper introduces Diffusion LAIR, a listwise preference optimization technique that leverages continuous reward scores instead of binary pairwise comparisons to align text‑to‑image diffusion models. LAIR transforms reward scores into centered advantage weights and optimizes an advantage‑weighted regression objective on an implicit reward defined by denoising‑loss improvement over a reference model, with a quadratic penalty to regulate reward magnitude. Experiments demonstrate that Diffusion LAIR surpasses strong baseline methods on SD1.5 and SDXL across generation, compositional, and editing tasks.
By Austin Wang, Jiaqi Han, Stefano Ermon, Yisong Yue
The paper introduces SatisDive, a training‑free inference method that balances reward and diversity in text‑to‑image diffusion by enforcing a reward floor for each image and a diversity cutoff for the batch. By adjusting the reward floor, the method traces a Pareto frontier between worst‑candidate reward and batch diversity. Experiments on Pick‑a‑Pic show that SatisDive consistently outperforms FK steering, improving worst‑candidate reward by up to 0.70 in some settings and Pareto‑dominating FK steering across overlapping DreamSim ranges.
By Kevin Zhai, Siva Rajesh Kasa, Soumya Roy, Sumit Negi, Mubarak Shah
arXiv:2604. 17415v3 Announce Type: replace-cross Abstract: Reward-based fine-tuning steers a pretrained diffusion or flow-based generative model toward higher-reward samples while remaining close to the pretrained model.
By Jeongjae Lee, Jinho Chang, Jeongsol Kim, Jong Chul Ye
arXiv:2512.10237v2 Announce Type: replace
Abstract: Reinforcement Learning from Human Feedback has emerged as a standard for aligning diffusion models. However, we identify a fundamental limitation i...
By Jiho Jang, Jinyoung Kim, Kyungjune Baek, Nojun Kwak
arXiv:2609.37372v1 Announce Type: new
Abstract: Visual reward models are essential for evaluating and improving visual generation models, yet existing approaches typically map task conditions and can...
By Xuehai Bai, Zhenchen Tang, Yang Shi, Dianyi Wang, Tengfei Liu, Wanshun Su, Xuanyu Zhu, Ruohui Wang, Haiwen Diao, Haotian Wang, Xiaoling Gu, Yuanxing Zhang
The paper introduces Objective-aware Trajectory Credit Assignment (OTCA), a framework that refines reinforcement learning for diffusion-based visual generation. OTCA decomposes credit across denoising steps and allocates multiple reward signals adaptively, addressing the coarse, uniform reward assignment of existing GRPO pipelines. Experiments demonstrate that OTCA consistently enhances image and video generation quality across various metrics.
By Rui Li, Ke Hao, Yuanzhi Liang, Haibin Huang, Chi Zhang, Yun Gu, Xuelong Li
arXiv:2607. 02291v1 Announce Type: new Abstract: Conventional reinforcement learning strategies for visual generation typically employ sample-wise reward functions, yet this practice frequently results in reward hacking that degrades image diversity and introduces visual anomalies.
By Ruihang Li, Mengde Xu, Shuyang Gu, Leigang Qu, Fuli Feng, Han Hu, Wenjie Wang
CoRe introduces a co‑evolving reward framework to mitigate latent reward hacking in video diffusion models. By continuously refitting the latent‑reward model on the generator’s current samples and anchoring it to real‑video preferences, CoRe prevents the generator from drifting outside the reward model’s training support. Experiments on Wan2.1‑T2V‑1.3B demonstrate that CoRe improves generation quality over pretrained models and prior alignment methods while avoiding quality collapse.
By Zhaolong Su, Yujin Han, Feng Wang, Jameson Dong, Hins Hu, Difan Zou
arXiv:2608.30194v1 Announce Type: new
Abstract: Diffusion models have recently advanced text-to-video (T2V) generation, yet they still struggle with fine-grained compositional alignment, such as attr...
By Yujiang Pu, Yu Kong
arXiv:2601.03468v2 Announce Type: replace
Abstract: Reinforcement learning (RL) has become a standard approach for post-training large language models and, more recently, for improving image generati...
By Yunqi Hong, Kuei-Chun Kao, Hengguang Zhou, Cho-Jui Hsieh
arXiv:2607. 15740v1 Announce Type: cross Abstract: As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important for fair and trustworthy generative AI.
By Bo-An Chang, Yu-Chih Chen