arXiv:2605. 17232v2 Announce Type: replace Abstract: Discrete diffusion has become a leading framework for generative modeling in various applications including language, vision, and biology.
By Kelvin Kan, Xingjian Li, Benjamin J. Zhang, Tuhin Sahai, Stanley Osher, Markos A. Katsoulakis
arXiv:2602. 15008v2 Announce Type: replace Abstract: Diffusion models over discrete spaces have recently shown striking empirical success, yet their theoretical foundations remain incomplete.
By Daniil Dmitriev, Zhihan Huang, Yuting Wei
arXiv:2610.07655v1 Announce Type: cross
Abstract: Discrete diffusion models have emerged as a practically successful framework for generative modeling on discrete product spaces, yet their statistica...
By Dongsun Yoon, Saptarshi Chakraborty
arXiv:2605. 00161v2 Announce Type: replace Abstract: Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive as high-quality samples still demand hundreds of refinement steps.
By Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song
The article "Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction" presents a unified primer on diffusion models that applies to both continuous Euclidean data and discrete categorical structures. It develops discrete-time forward noising via Markov kernels and learned reverse dynamics, and connects these to continuous-time limits such as stochastic differential equations in ρ^d and continuous-time Markov chains on finite alphabets, deriving the corresponding Fokker–Planck and master equations. The work also shows how different forward corruption choices—Gaussian processes for continuous spaces and structured categorical transition kernels for discrete spaces—affect reverse dynamics and the evidence lower bound used in training, offering a layered exposition for newcomers, practitioners, and experts alike.
By Vincent Pauline, Tobias H\"oppe, Kirill Neklyudov, Alexander Tong, Stefan Bauer, Andrea Dittadi
LangFlow is a continuous diffusion language model that bridges embedding-space diffusion with Flow Matching, introducing three key innovations: an ODE-based NLL bound, an information-uniform noise schedule using a learnable Gumbel distribution, and self-conditioning training. These advances enable LangFlow to match or surpass top discrete diffusion models in generative quality, achieving perplexities of 92.3 on LM1B and 36.5 on OpenWebText, and outperforming discrete counterparts on reasoning tasks such as Sudoku. The work demonstrates that continuous diffusion can rival discrete diffusion for language modeling, offering a promising new paradigm.
By Yuxin Chen, Chumeng Liang, Hangke Sui, Ruihan Guo, Chaoran Cheng, Jiaxuan You, Ge Liu