The paper presents a theoretical analysis of score mismatch in discrete diffusion models, deriving a KL divergence upper bound that quantifies the bias introduced when the training and target distributions differ. It examines partial-observation constraints such as text infilling and sentence completion, showing that the Conditional SEDD sampler can incur a bias that grows with dimension, vocabulary size, and compatibility. To mitigate this, the authors propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints, demonstrating through experiments that it effectively reduces bias and improves sampling performance.
By Renxiang Huang, Yuchen Liang, Yingbin Liang, Ness Shroff, Lifeng Lai
arXiv:2607. 23226v1 Announce Type: new Abstract: Despite the empirical success of score-based diffusion models, a complete theoretical understanding of how finite-sample learning, network parameterization, and numerical discretization jointly dictate generative quality remains underdeveloped.
By Jinshu Huang, Yiming Jiang, Chunlin Wu
arXiv:2602. 15008v2 Announce Type: replace Abstract: Diffusion models over discrete spaces have recently shown striking empirical success, yet their theoretical foundations remain incomplete.
By Daniil Dmitriev, Zhihan Huang, Yuting Wei
arXiv:2610.19020v1 Announce Type: cross
Abstract: Discrete diffusion models have emerged as a leading alternative to autoregressive models for generating text and other categorical data, yet their th...
By Luca Blunda, Tyler Farghly, Umut Simsekli, Alain Durmus
arXiv:2501. 12982v3 Announce Type: replace-cross Abstract: This paper investigates how diffusion generative models leverage (unknown) low-dimensional structure to accelerate sampling.
By Jiadong Liang, Zhihan Huang, Yuxin Chen
arXiv:2607. 04442v1 Announce Type: cross Abstract: Diffusion models (DMs) are a state-of-the-art generative method to approximately sample from an unknown distribution.
By Benjamin Dupuis, Tyler Farghly, Maxime Haddouche, Alain Durmus, Umut Simsekli
arXiv:2607. 15485v1 Announce Type: new Abstract: Score-based generative models exhibit a puzzling behavior: they often appear to cover all modes of a target multimodal distribution and yet may fail to learn the correct relative mode amplitudes, which can be interpreted as mixture weights.
By Andrew Dennehy, Ramchandran Muthukumar, Rebecca Willett, Nisha Chandramoorthy
arXiv:2504.05161v2 Announce Type: replace-cross
Abstract: Score estimation is the backbone of score-based generative models (SGMs), especially denoising diffusion probabilistic models (DDPMs). A key...
By Sinho Chewi, Alkis Kalavasis, Anay Mehrotra, Omar Montasser
arXiv:2609.27546v1 Announce Type: cross
Abstract: Score-based diffusion models are increasingly considered in settings where the underlying data distribution may differ from the training distribution...
By Wei Luo, Neil K. Chada, Shijie Zhang, Lu Yu
arXiv:2511. 05940v3 Announce Type: replace-cross Abstract: Score-based diffusion models have emerged as a powerful class of generative methods, with successful applications across diverse domains.
By Kang Liu, Enrique Zuazua
The paper presents sample complexity bounds for sampling high‑dimensional categorical distributions with local dependence using discrete diffusions. It introduces a pinning decomposition of the discrete score, enabling a weight‑sharing neural score learner combined with τ‑leaping for efficient end‑to‑end sampling. The authors derive explicit guarantees that depend on vocabulary size, MRF interaction order, and sample size, and demonstrate that weight‑sharing networks outperform fully connected ones on Potts, Ising, and tree‑structured models.
By Shivam Kumar, Nabarun Deb
arXiv:2512. 24152v2 Announce Type: replace-cross Abstract: Sampling based on score diffusions has led to striking empirical results, and has attracted considerable attention from various research communities.
By M. J. Wainwright