Hugging Face Trending Papers
Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 15
Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
arXiv:2610.19042v1 Announce Type: new Abstract: Understanding the mechanism behind why diffusion models memorize their training data remains an open question in generative modeling. In this paper, we...
arXiv Machine Learning
Oct 9
Beyond Sample Copying: Structural Memorization in Diffusion Models
The paper investigates why diffusion models, despite their practical success, can fully memorize training data when optimal. It shows that as training proceeds, the denoising objective causes overfitting at intermediate noise levels, creating a gap between validation and training performance. In a 2D toy model, this gap arises from the interplay between model error and the data distribution’s support density, leading to a smooth, generalizing flow field rather than exact recall of training points. The study also reveals that predictions from intermediate inference states become increasingly similar to training data in feature space, even as overall similarity to validation data remains unchanged.
arXiv Machine Learning
Jun 10
The Emergence of Reproducibility and Generalizability in Diffusion Models
arXiv:2310. 05264v5 Announce Type: replace Abstract: In this work, we investigate an intriguing and prevalent phenomenon of diffusion models which we term as "consistent model reproducibility": given the same starting noise input and a deterministic sampler, different diffusion models often yield remarkably similar outputs.
arXiv Machine Learning
Oct 9
Early Signatures of Memorization in Diffusion Models via Basin Geometry and Cyclic Denoising
The paper demonstrates that diffusion models encode memorization in the geometry of their learned energy landscape before it becomes visible in generated samples. By analyzing score divergence and basin volume, the authors show that localized basins form around training images early in training, and that cyclic denoising can recover these images even when one‑shot generation shows no copies. Experiments on Gaussian mixtures, CelebA, CIFAR‑10, and Stable Diffusion v1.4 confirm that basin geometry reveals memorization across various settings.
arXiv AI
Sep 3
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
The paper investigates when language diffusion models, specifically Uniform-based Discrete Diffusion Models (UDDMs), shift from memorizing training data to generalizing to new data. It shows that UDDMs act as associative memories, forming basins of attraction around stored examples without requiring an explicit energy function. By measuring token recovery and conditional entropy, the authors identify a sharp transition governed by training set size, where memorization (vanishing entropy) gives way to generalization (finite entropy).
arXiv AI
Oct 1
Distilling Diffusion Score Discrepancy for Efficient Training Data Attribution
The paper introduces a new method for training data attribution in diffusion models called TID, which uses a local score discrepancy measure and can be estimated without retraining. It further distills this approach into TIDE, a forward‑only student that reproduces the teacher’s rankings using internal activations, achieving comparable accuracy at dramatically lower query cost. Experiments on CIFAR‑10, ArtBench‑10, and MS‑COCO show that TID outperforms existing methods and TIDE attributes samples in milliseconds, faster than generation itself.