← Back to all news
arXiv Machine Learning By Hugo Latourelle-Vigeant, Sinho Chewi, Aram-Alexandre Pooladian, John Sous, Theodor Misiakiewicz

Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regime

Read the original on arXiv Machine Learning →

The paper investigates diffusion models trained in a lazy high‑dimensional regime, extending benign overfitting theory to generative settings. By analyzing denoising score matching in a vector‑valued RKHS with an inner‑product kernel, the authors derive exact risk trajectories under gradient flow when the number of samples scales proportionally with dimensionality. These trajectories reveal three distinct phases—spectral generalization, noise‑dominated interpolation, and empirical Bayes memorization—whose interplay shapes the distribution of generated samples.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
3d ago

Beyond Sample Copying: Structural Memorization in Diffusion Models

The paper investigates why diffusion models, despite their practical success, can fully memorize training data when optimal. It shows that as training proceeds, the denoising objective causes overfitting at intermediate noise levels, creating a gap between validation and training performance. In a 2D toy model, this gap arises from the interplay between model error and the data distribution’s support density, leading to a smooth, generalizing flow field rather than exact recall of training points. The study also reveals that predictions from intermediate inference states become increasingly similar to training data in feature space, even as overall similarity to validation data remains unchanged.

By Tim Kaiser, Markus Kollmann
diffusion
More like this →
arXiv Machine Learning
Jun 9

Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles

arXiv:2606. 09718v1 Announce Type: new Abstract: Diffusion models have demonstrated remarkable generative capabilities and have also emerged as powerful self-supervised representation learners, yet the connection between these two abilities remains less explored.

By Xiao Li, Yixuan Jia, Zekai Zhang, Xiang Li, Lianghe Shi, Jinxin Zhou, Zhihui Zhu, Liyue Shen, Qing Qu
diffusion
More like this →
arXiv Machine Learning
Jul 7

Benign Overfitting Does Not Occur in Diffusion Models

arXiv:2607. 02671v1 Announce Type: cross Abstract: Benign overfitting and double descent have come to shape our understanding of generalization in deep learning, establishing that overfitting is not only compatible with good generalization but can actively benefit it.

By Tyler Farghly, Benjamin Dupuis, Alain Durmus, Umut Simsekli
diffusionsafety
More like this →