The paper investigates why diffusion models, despite their practical success, can fully memorize training data when optimal. It shows that as training proceeds, the denoising objective causes overfitting at intermediate noise levels, creating a gap between validation and training performance. In a 2D toy model, this gap arises from the interplay between model error and the data distribution’s support density, leading to a smooth, generalizing flow field rather than exact recall of training points. The study also reveals that predictions from intermediate inference states become increasingly similar to training data in feature space, even as overall similarity to validation data remains unchanged.
By Tim Kaiser, Markus Kollmann
The paper identifies a limitation of discrete graph diffusion models: independent edge updates combined with exchangeability force the terminal prior to be an Erdős–Rényi graph, which hampers denoising and leads to loss of node heterogeneity. To overcome this, the authors propose a Latent Sociability Prior (LSP) and a joint diffusion process that co‑evolves latent sociability variables with graph structure, eliminating edge independence while maintaining exchangeability. Experiments on several graph generation datasets show that this approach better captures graph distributions than baseline methods that lack node features.
By Adarsh Jamadandi, Nicolas Keriven, Aline Roumy
The paper demonstrates that diffusion models encode memorization in the geometry of their learned energy landscape before it becomes visible in generated samples. By analyzing score divergence and basin volume, the authors show that localized basins form around training images early in training, and that cyclic denoising can recover these images even when one‑shot generation shows no copies. Experiments on Gaussian mixtures, CelebA, CIFAR‑10, and Stable Diffusion v1.4 confirm that basin geometry reveals memorization across various settings.
By Nikhil Verma, Siddharthan Dileep, Anoop Singh, Srikanth Sastry, Ramya Hebbalaguppe, Sayan Ranu, N. M. Anoop Krishnan
The paper introduces the Graph Dynamics Model (GDM), a world model that learns stochastic latent dynamics over evolving graph topologies. GDM employs a sparse recurrent adjacency matrix for topology updates and a recurrent state‑space architecture for stochastic transitions, enabling it to handle partially observable, stochastic environments. The authors also propose the Graph Distribution Distance (GDD) metric, using maximum mean discrepancy with a graph kernel, to compare predicted and true joint graph state distributions, and demonstrate GDM’s superior performance and zero‑shot generalisation on large graphs.
By Alex Schutz, Nick Hawes, Victor-Alexandru Darvariu
arXiv:2606. 02241v1 Announce Type: new Abstract: Is the uniform-state diffusion framework a more powerful paradigm for discrete diffusion?
By Justin Deschenaux, Caglar Gulcehre
arXiv:2609.39658v1 Announce Type: new
Abstract: Diffusion models generate data by reversing a forward corruption process that typically approaches a simple Gaussian prior. Recent work has extended th...
By Jinwei Li, Daniel Tenbrinck
arXiv:2607. 07232v1 Announce Type: cross Abstract: Diffusion models represent a leading paradigm for graph generation, with notable impact in domains such as molecular design.
By Sergio Rozada, Yiming Qin, Manuel Madeira, Pascal Frossard, Alejandro Ribeiro
Diffusion models represent a leading paradigm for graph generation, with notable impact in domains such as molecular design. Yet, scaling these models to large graphs remains an open problem.
The paper examines how Denoising Diffusion Probabilistic Models (DDPMs) perform on globally constrained discrete tasks such as Sudoku and N-queens. It shows that standard diffusion sampling, which keeps updates close to the noisy state, often preserves early mistakes, whereas sampling directly from the model’s clean predictions dramatically improves validity (e.g., Sudoku from 31% to 95%). The authors further introduce self‑correction training, exposing the model to its own predictions to reduce inference errors, which enhances the performance of standard samplers across tasks.
By Mariia Drozdova, St\'ephane Liem Nguyen, Fran\c{c}ois Fleuret
arXiv:2609.37974v1 Announce Type: cross
Abstract: Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The m...
By Manuel Madeira, Amitis Shidani, Alice Bizeul, Victor Turrisi, Louis B\'ethune, Bhavika Devnani, Dan Busbridge, Pierre Ablin, Jo\~ao Monteiro
arXiv:2610.19042v1 Announce Type: new
Abstract: Understanding the mechanism behind why diffusion models memorize their training data remains an open question in generative modeling. In this paper, we...
By Shashank Sule, Xinyue Yu, Boonam Shin, Yuxuan Liu, Abolfazl Hashemi, Hayden Schaeffer
arXiv:2507. 14484v2 Announce Type: replace Abstract: In recent years, graph neural networks (GNN) have achieved unprecedented successes in node classification tasks.
By Yule Li, Yifeng Lu, Zhen Wang, Zhewei Wei, Yaliang Li, Bolin Ding