The paper investigates why diffusion models, despite their practical success, can fully memorize training data when optimal. It shows that as training proceeds, the denoising objective causes overfitting at intermediate noise levels, creating a gap between validation and training performance. In a 2D toy model, this gap arises from the interplay between model error and the data distribution’s support density, leading to a smooth, generalizing flow field rather than exact recall of training points. The study also reveals that predictions from intermediate inference states become increasingly similar to training data in feature space, even as overall similarity to validation data remains unchanged.
By Tim Kaiser, Markus Kollmann
The paper identifies a limitation of discrete graph diffusion models: independent edge updates combined with exchangeability force the terminal prior to be an Erdős–Rényi graph, which hampers denoising and leads to loss of node heterogeneity. To overcome this, the authors propose a Latent Sociability Prior (LSP) and a joint diffusion process that co‑evolves latent sociability variables with graph structure, eliminating edge independence while maintaining exchangeability. Experiments on several graph generation datasets show that this approach better captures graph distributions than baseline methods that lack node features.
By Adarsh Jamadandi, Nicolas Keriven, Aline Roumy
The paper demonstrates that diffusion models encode memorization in the geometry of their learned energy landscape before it becomes visible in generated samples. By analyzing score divergence and basin volume, the authors show that localized basins form around training images early in training, and that cyclic denoising can recover these images even when one‑shot generation shows no copies. Experiments on Gaussian mixtures, CelebA, CIFAR‑10, and Stable Diffusion v1.4 confirm that basin geometry reveals memorization across various settings.
By Nikhil Verma, Siddharthan Dileep, Anoop Singh, Srikanth Sastry, Ramya Hebbalaguppe, Sayan Ranu, N. M. Anoop Krishnan
The paper introduces the Graph Dynamics Model (GDM), a world model that learns stochastic latent dynamics over evolving graph topologies. GDM employs a sparse recurrent adjacency matrix for topology updates and a recurrent state‑space architecture for stochastic transitions, enabling it to handle partially observable, stochastic environments. The authors also propose the Graph Distribution Distance (GDD) metric, using maximum mean discrepancy with a graph kernel, to compare predicted and true joint graph state distributions, and demonstrate GDM’s superior performance and zero‑shot generalisation on large graphs.
By Alex Schutz, Nick Hawes, Victor-Alexandru Darvariu
arXiv:2606. 02241v1 Announce Type: new Abstract: Is the uniform-state diffusion framework a more powerful paradigm for discrete diffusion?
By Justin Deschenaux, Caglar Gulcehre
arXiv:2609.39658v1 Announce Type: new
Abstract: Diffusion models generate data by reversing a forward corruption process that typically approaches a simple Gaussian prior. Recent work has extended th...
By Jinwei Li, Daniel Tenbrinck