arXiv Machine Learning

Blind denoising diffusion models and the blessings of dimensionality

arXiv:2602. 09639v2 Announce Type: replace Abstract: Denoising diffusion models (DDMs) are state-of-the-art methods for learning densities from data across numerous domains, yet many aspects of the training and sampling pipeline remain poorly understood.

arXiv Machine Learning
Aug 31

Diffusion models as plug-and-play priors

The paper explores using denoising diffusion generative models as plug‑and‑play priors for high‑dimensional inference problems. By combining a pre‑trained diffusion prior with a differentiable auxiliary constraint, the authors enable approximate inference through iterative differentiation across multiple noisy versions of the data. This framework opens possibilities for conditional generation, image segmentation, and novel combinatorial optimization algorithms.

By Alexandros Graikos, Esmeralda S. Whitammer, Nebojsa Jojic, Dimitris Samaras
arXiv Machine Learning
Sep 10

Noise in Diffusion Models Is a Learnable Input

The paper argues that the concrete random noise used in diffusion models is not merely a passive perturbation but a learnable input that can be exploited by the model. By analyzing how clean data and realized noise jointly form the noisy input, the authors show that the model can learn regularities in the data or in the noise structure, and that these two routes can interact. Experiments on MNIST and CIFAR‑10 using pseudorandom streams demonstrate that structured‑noise training can reduce prediction loss, but this advantage disappears when test noise is replaced with IID noise, indicating that the learned dependence is tied to the specific noise structure.

By Shengzhi Deng, Chenqi Ye, Yanze Guo
arXiv Machine Learning
Jun 10

The Emergence of Reproducibility and Generalizability in Diffusion Models

arXiv:2310. 05264v5 Announce Type: replace Abstract: In this work, we investigate an intriguing and prevalent phenomenon of diffusion models which we term as "consistent model reproducibility": given the same starting noise input and a deterministic sampler, different diffusion models often yield remarkably similar outputs.

By Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, Qing Qu
arXiv Machine Learning
Aug 10

Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions

arXiv:2409. 18804v3 Announce Type: replace-cross Abstract: Denoising Diffusion Probabilistic Models (DDPM) are powerful state-of-the-art methods used to generate synthetic data from high-dimensional data distributions and are widely used for image, audio, and video generation as well as many more applications in science and beyond.

By Iskander Azangulov, George Deligiannidis, Judith Rousseau
arXiv Machine Learning
Aug 31

Improved off-policy training of diffusion samplers

The paper investigates training diffusion models to sample from distributions defined by unnormalized densities or energy functions. It benchmarks various diffusion-structured inference techniques, including simulation-based variational methods and off-policy approaches such as continuous generative flow networks, highlighting their relative strengths and challenging some prior claims. Additionally, the authors introduce a new exploration strategy for off-policy methods that employs local search in the target space with a replay buffer, demonstrating improved sample quality across multiple target distributions.

By Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, Esmeralda S. Whitammer