The paper investigates memorization in diffusion models, showing that the empirical score function is a weighted sum of Gaussian score functions with sharp softmax weights, causing individual training samples to dominate and lead to sampling collapse. By approximating this function with a neural network, the authors obtain a smoother representation that generalizes better. They introduce two techniques—Noise Unconditioning and Temperature Smoothing—to further reduce single‑sample dominance, and demonstrate improved generalization across multiple datasets while preserving generation quality.
By Xinyu Zhou, Jiawei Zhang, Stephen J. Wright
arXiv:2603. 22962v3 Announce Type: replace Abstract: We study the theoretical behavior of denoising score matching--the learning task associated to diffusion models--when the data distribution is supported on a low-dimensional manifold and the score is parameterized using a random feature neural network.
By Anand Jerry George, Nicolas Macris
The paper investigates diffusion models trained in a lazy high‑dimensional regime, extending benign overfitting theory to generative settings. By analyzing denoising score matching in a vector‑valued RKHS with an inner‑product kernel, the authors derive exact risk trajectories under gradient flow when the number of samples scales proportionally with dimensionality. These trajectories reveal three distinct phases—spectral generalization, noise‑dominated interpolation, and empirical Bayes memorization—whose interplay shapes the distribution of generated samples.
By Hugo Latourelle-Vigeant, Sinho Chewi, Aram-Alexandre Pooladian, John Sous, Theodor Misiakiewicz
arXiv:2509. 24710v2 Announce Type: replace-cross Abstract: Score-based diffusion models are a highly effective method for generating samples from a distribution of images.
By Dennis Elbr\"achter, Giovanni S. Alberti, Matteo Santacesaria
arXiv:2605. 16415v3 Announce Type: replace-cross Abstract: The creativity of diffusion models refers to their ability to generate highly realistic images that are different from their training data.
By Itamar Levine, Yair Weiss
arXiv:2606. 19894v1 Announce Type: new Abstract: The remarkable success of score-based diffusion models has spurred significant efforts to establish their theoretical foundations.
By Xinhe Mu, Zaijiu Shang, Zhaoqi Zhou, Chuan Zhou, Qi Meng, Guiying Yan, Zhiming Ma
arXiv:2606. 08309v1 Announce Type: new Abstract: Score-based generative models have had remarkable success over the last decade in generating a diverse set of visually plausible images.
By Emma Finn, Binxu Wang, T. Anderson Keller, Demba E. Ba
arXiv:2409. 18804v3 Announce Type: replace-cross Abstract: Denoising Diffusion Probabilistic Models (DDPM) are powerful state-of-the-art methods used to generate synthetic data from high-dimensional data distributions and are widely used for image, audio, and video generation as well as many more applications in science and beyond.
By Iskander Azangulov, George Deligiannidis, Judith Rousseau
arXiv:2502. 00336v3 Announce Type: replace Abstract: We theoretically investigate the phenomena of generalization and memorization in diffusion models.
By Anand Jerry George, Rodrigo Veiga, Nicolas Macris
The book "The Principles of Diffusion Models" outlines the foundational concepts behind diffusion models, tracing their evolution from a forward process that corrupts data into noise to a reverse process that reconstructs data. It presents three complementary perspectives—variational, score-based, and flow-based—each describing how a time-dependent velocity field transports a simple prior to the data distribution. The text also covers practical guidance for controllable generation, efficient solvers, and diffusion-inspired flow-map models, providing a mathematically grounded framework for readers with basic deep‑learning knowledge.
By Chieh-Hsin Lai, Yang Song, Dongjun Kim, Yuki Mitsufuji, Stefano Ermon
arXiv:2607. 04775v1 Announce Type: cross Abstract: Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications.
By Stanislas Strasman (SU, LPSM), Sobihan Surendran (SU, LPSM), Sylvain Le Corff (SU, LPSM)
Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications. While the statistical properties of their sampling procedures are increasingly well understood, the optimization dynamics underlying their training remain less explored.