arXiv AI
The critical slowing down in training diffusion models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 10
The Emergence of Reproducibility and Generalizability in Diffusion Models
arXiv:2310. 05264v5 Announce Type: replace Abstract: In this work, we investigate an intriguing and prevalent phenomenon of diffusion models which we term as "consistent model reproducibility": given the same starting noise input and a deterministic sampler, different diffusion models often yield remarkably similar outputs.
arXiv Machine Learning
Aug 4
Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping
arXiv:2608. 01833v1 Announce Type: cross Abstract: Grokking is a striking phenomenon in neural network training, where a model can undergo a prolonged period of pure memorization before abrupt generalization.
arXiv Machine Learning
Jul 28
From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models
arXiv:2607. 23226v1 Announce Type: new Abstract: Despite the empirical success of score-based diffusion models, a complete theoretical understanding of how finite-sample learning, network parameterization, and numerical discretization jointly dictate generative quality remains underdeveloped.
arXiv Machine Learning
Jun 15
Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization
arXiv:2606. 13796v1 Announce Type: cross Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution.
arXiv Machine Learning
Sep 11
Smoothing the Score Function to Enhance Generalization in Diffusion Models
The paper investigates memorization in diffusion models, showing that the empirical score function is a weighted sum of Gaussian score functions with sharp softmax weights, causing individual training samples to dominate and lead to sampling collapse. By approximating this function with a neural network, the authors obtain a smoother representation that generalizes better. They introduce two techniques—Noise Unconditioning and Temperature Smoothing—to further reduce single‑sample dominance, and demonstrate improved generalization across multiple datasets while preserving generation quality.
Hugging Face Trending Papers
Jul 21
ROMS-IMLE: A Minimalist Approach to Competitive Single-Step Generative Modelling
Generative models have undergone many generations of evolution, from VAEs/GANs to diffusion/flow matching. Along the way, the underlying techniques have become more complicated and various beliefs about what drives strong empirical performance have taken hold.