arXiv AI June 4, 2026 By Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson, Lukasz Golab

Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences

arXiv:2605. 07724v2 Announce Type: replace-cross Abstract: Recursive retraining of generative models poses a critical representation challenge: when synthetic outputs are curated based on a fixed reward signal, the model tends to collapse onto a narrow set of outputs that over-optimize that objective.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

safety

arXiv Machine Learning

Jul 3

Optimizing Visual Generative Models via Distribution-wise Rewards

arXiv:2607. 02291v1 Announce Type: new Abstract: Conventional reinforcement learning strategies for visual generation typically employ sample-wise reward functions, yet this practice frequently results in reward hacking that degrades image diversity and introduces visual anomalies.

By Ruihang Li, Mengde Xu, Shuyang Gu, Leigang Qu, Fuli Feng, Han Hu, Wenjie Wang

reinforcement-learning safety

arXiv AI

Jun 3

Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning

arXiv:2606. 03962v1 Announce Type: cross Abstract: Classical reinforcement learning (RL) typically seeks a deterministic policy that maximizes the expected sum of a scalar reward.

By Anthony GX-Chen, Ankit Anand, Gheorghe Comanici, Zaheer Abbas, Eser Ayg\"un, David Smalling, Shibl Mourad, Doina Precup, Andr\'e Barreto, Mark Rowland

llms agents reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 2

The Representation-Rationalizability Tradeoff in Reward Learning

arXiv:2606. 00291v1 Announce Type: cross Abstract: In RLHF, each training example contains a prompt $x$ and two candidate responses $y,y'$, and annotators provide pairwise preferences between these responses.

By Jing Dong, Yaoliang Yu, Pascal Pourpart

rag reinforcement-learning

arXiv Machine Learning

Jul 2

Generative Refinement for Low-Budget Black-Box Optimization

arXiv:2607. 00691v1 Announce Type: new Abstract: Black-box optimization is a fundamental science and engineering tool that makes it possible to optimize objectives without gradient information.

By Edouard R. Dufour, Pascal Fua

Hugging Face Trending Papers

Jun 2

Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning

Classical reinforcement learning (RL) typically seeks a deterministic policy that maximizes the expected sum of a scalar reward. Yet, modern applications such as language model fine-tuning or scientific discovery demand diversity.

llms agents reinforcement-learning fine-tuning

arXiv AI

Aug 6

Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation

arXiv:2601. 12401v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has emerged as a powerful paradigm for fine-tuning large-scale generative models, such as diffusion and flow models, to align with complex human preferences and user-specified tasks.

By Jinmei Liu, Haoru Li, Zhenhong Sun, Chaofeng Chen, Yatao Bian, Bo Wang, Daoyi Dong, Chunlin Chen, Zhi Wang

llms diffusion reinforcement-learning fine-tuning safety

Related stories

Optimizing Visual Generative Models via Distribution-wise Rewards

Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning

The Representation-Rationalizability Tradeoff in Reward Learning

Generative Refinement for Low-Budget Black-Box Optimization

Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning

Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation