arXiv:2607. 26285v1 Announce Type: cross Abstract: Two central challenges in diffusion-based sampling are the theoretical one of understanding their remarkable effectiveness even in high-dimensional settings, and the practical one of designing algorithms with certified performance guarantees.
By Martin J. Wainwright
arXiv:2608.28949v1 Announce Type: cross
Abstract: We study a class of product-reference diffusion algorithms for sampling from a discrete distribution. We show that their sampling performance can be...
By Martin J. Wainwright
arXiv:2602. 15008v2 Announce Type: replace Abstract: Diffusion models over discrete spaces have recently shown striking empirical success, yet their theoretical foundations remain incomplete.
By Daniil Dmitriev, Zhihan Huang, Yuting Wei
The paper introduces a new framework for adaptive parallel sampling of discrete vectors, where a deterministic policy reveals coordinates round‑by‑round based on previously observed values and samples the remaining coordinates from their exact conditional marginals. The authors prove an exact identity linking the forward Kullback‑Leibler divergence of any policy to the expected conditional total correlation accumulated during the sampling process, establishing conditional total correlation as the precise information cost of within‑round parallelism. Using this identity, they derive zero‑error schedules for finite‑order Markov chains, characterize the serial depth of Bernoulli walks, and demonstrate separations between different reveal orders, permutation strategies, and string structures, thereby revealing how conditional dependence governs parallelizability.
whyItMatters":"The results provide a principled, information‑theoretic measure of parallel sampling efficiency that can guide the design of decoding rules for masked diffusion models and other generative systems."
By Chuling Wen, Weijie Liang, Jian Lu
The paper studies how to choose sampling schedules for tau‑leaping in masked discrete diffusion models. By deriving an exact integral representation of the factorization error ε_fact in terms of a dependence density ρ, the authors develop estimators and recursive equations that identify the unique optimal schedule under a monotonicity condition. In the large‑scale limit, they provide explicit characterizations of the optimal smooth schedule and show that while optimizing smooth schedules can improve constants, it does not change the N/K scaling unless the dependence density degenerates, in which case asymptotic improvements are possible.
By Cecilia Secchi, Giacomo Zanella
The paper proves that for discrete diffusion models using uniform or remasking forward processes, an adaptive sampler based on a leave‑one‑out denoiser can achieve sampling error proportional to the score‑estimation error plus a small tolerance. The required number of discretization steps scales with the dual total correlation of the target distribution, not directly with the ambient dimension. This result shows that sampling complexity is governed by the intrinsic dependence structure of the distribution, and the authors provide an information‑theoretic analysis linking discretization error to mutual information between coordinates.
By Daniil Dmitriev, Zhihan Huang, Yuting Wei