← Back to all news
arXiv Machine Learning By Nail B. Khelifa, Richard E. Turner, Ramji Venkataramanan

Quantifying Error Propagation and Model Collapse in Diffusion Models

Read the original on arXiv Machine Learning →

The paper investigates how repeatedly training score-based diffusion models on synthetic data can cause their outputs to drift from the target distribution. It provides theoretical upper and lower bounds on the divergence accumulated over successive training rounds, distinguishing regimes based on score estimation error and the fraction of fresh data used. Empirical experiments on synthetic data and images support the theoretical findings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 10

The Emergence of Reproducibility and Generalizability in Diffusion Models

arXiv:2310. 05264v5 Announce Type: replace Abstract: In this work, we investigate an intriguing and prevalent phenomenon of diffusion models which we term as "consistent model reproducibility": given the same starting noise input and a deterministic sampler, different diffusion models often yield remarkably similar outputs.

By Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, Qing Qu
diffusionfine-tuningsafety
More like this →
arXiv AI
Jun 24

Catastrophic Compositional Generation: Why Vanilla Diffusion Models Fail to Extrapolate

arXiv:2606. 23920v1 Announce Type: cross Abstract: The task of compositional generation involves using a conditional generative model, trained only on a subset of the possible conditions, to produce samples from compositionally-defined target distributions such as a geometric combination of the source distributions.

By Duncan Soiffer, Chandler Squires, Yuan Guan, Jason Hartford, Pradeep Ravikumar
diffusion
More like this →
arXiv AI
Oct 1

Distilling Diffusion Score Discrepancy for Efficient Training Data Attribution

The paper introduces a new method for training data attribution in diffusion models called TID, which uses a local score discrepancy measure and can be estimated without retraining. It further distills this approach into TIDE, a forward‑only student that reproduces the teacher’s rankings using internal activations, achieving comparable accuracy at dramatically lower query cost. Experiments on CIFAR‑10, ArtBench‑10, and MS‑COCO show that TID outperforms existing methods and TIDE attributes samples in milliseconds, faster than generation itself.

By Shixuan Liu, Joan Serr\`a, Kin Wai Cheuk, Jinju Kim, Woosung Choi, Yukara Ikemiya, Wei-Hsiang Liao, Jiaqi W. Ma, Yuki Mitsufuji
diffusionbenchmarks
More like this →
arXiv Machine Learning
Oct 9

Beyond Sample Copying: Structural Memorization in Diffusion Models

The paper investigates why diffusion models, despite their practical success, can fully memorize training data when optimal. It shows that as training proceeds, the denoising objective causes overfitting at intermediate noise levels, creating a gap between validation and training performance. In a 2D toy model, this gap arises from the interplay between model error and the data distribution’s support density, leading to a smooth, generalizing flow field rather than exact recall of training points. The study also reveals that predictions from intermediate inference states become increasingly similar to training data in feature space, even as overall similarity to validation data remains unchanged.

By Tim Kaiser, Markus Kollmann
diffusion
More like this →