arXiv Statistics ML

KL Convergence Guarantees for Score diffusion models under minimal data assumptions

The paper presents new Kullback–Leibler convergence guarantees for score diffusion models that avoid restrictive Lipschitz assumptions on the score function. By analyzing models with fixed step size derived from the Ornstein–Uhlenbeck semigroup and its kinetic counterpart, the authors derive simple, sharp KL bounds that hold for any data distribution with finite Fisher information relative to the standard Gaussian. This extends theoretical guarantees to a broader class of data distributions without requiring early stopping or compact support.

arXiv Machine Learning
Jul 17

The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis

arXiv:2506. 11378v3 Announce Type: replace Abstract: Sampling in score-based diffusion models can be performed by solving either a reverse-time stochastic differential equation (SDE) parameterized by an arbitrary stochasticity function or a probability flow ODE, corresponding to setting this stochasticity function to zero.

By Bernardo P. Schaeffer, Ricardo M. S. Rosa, Glauco Valle
arXiv Statistics ML
Sep 28

First-Order Stationarity of Reverse Diffusions

The paper establishes a first‑order theoretical framework for diffusion models, showing that SDE‑based reverse‑time flows of both overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates when the stationary potential of the forward process is strongly convex. It further incorporates discretization to provide averaged first‑order stationarity bounds—sampling analogues of averaged gradient‑norm guarantees in nonconvex optimization—for samplers of both diffusion models. These results highlight a unique advantage of SDE‑based reverse diffusion over ODE‑based approaches, offering local convexity‑free certificates that ensure score consistency rather than global mode weights.

By Zhifeng Chen, Chenyang Jiang, Yazhen Wang
arXiv Machine Learning
2d ago

Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models

The paper establishes finite‑sample guarantees for fully discrete conditional diffusion models trained from data and run with a finite number of denoising steps. By aligning the score‑training objective with the sampling grid, the authors jointly analyze statistical estimation and numerical discretization errors. For any smoothness index β>0, they prove that an appropriately chosen grid yields an averaged conditional Kullback–Leibler divergence of ˗O_p(n^{-2β/(d_x+d_y+2β)}) and a total‑variation error of ˗O_p(n^{-β/(d_x+d_y+2β)}), matching known rates up to logarithmic factors.

By Qiuyun Zhu