The paper investigates memorization in diffusion models, showing that the empirical score function is a weighted sum of Gaussian score functions with sharp softmax weights, causing individual training samples to dominate and lead to sampling collapse. By approximating this function with a neural network, the authors obtain a smoother representation that generalizes better. They introduce two techniques—Noise Unconditioning and Temperature Smoothing—to further reduce single‑sample dominance, and demonstrate improved generalization across multiple datasets while preserving generation quality.
By Xinyu Zhou, Jiawei Zhang, Stephen J. Wright
arXiv:2603. 22962v3 Announce Type: replace Abstract: We study the theoretical behavior of denoising score matching--the learning task associated to diffusion models--when the data distribution is supported on a low-dimensional manifold and the score is parameterized using a random feature neural network.
By Anand Jerry George, Nicolas Macris
The paper investigates diffusion models trained in a lazy high‑dimensional regime, extending benign overfitting theory to generative settings. By analyzing denoising score matching in a vector‑valued RKHS with an inner‑product kernel, the authors derive exact risk trajectories under gradient flow when the number of samples scales proportionally with dimensionality. These trajectories reveal three distinct phases—spectral generalization, noise‑dominated interpolation, and empirical Bayes memorization—whose interplay shapes the distribution of generated samples.
By Hugo Latourelle-Vigeant, Sinho Chewi, Aram-Alexandre Pooladian, John Sous, Theodor Misiakiewicz
arXiv:2509. 24710v2 Announce Type: replace-cross Abstract: Score-based diffusion models are a highly effective method for generating samples from a distribution of images.
By Dennis Elbr\"achter, Giovanni S. Alberti, Matteo Santacesaria
arXiv:2605. 16415v3 Announce Type: replace-cross Abstract: The creativity of diffusion models refers to their ability to generate highly realistic images that are different from their training data.
By Itamar Levine, Yair Weiss
arXiv:2606. 19894v1 Announce Type: new Abstract: The remarkable success of score-based diffusion models has spurred significant efforts to establish their theoretical foundations.
By Xinhe Mu, Zaijiu Shang, Zhaoqi Zhou, Chuan Zhou, Qi Meng, Guiying Yan, Zhiming Ma