arXiv:2608.25505v2 Announce Type: replace-cross
Abstract: Masked diffusion models generate sequences in rounds, predicting several masked positions at once. Fewer rounds can accelerate decoding, but...
By Chuling Wen, Weijie Liang, Jian Lu, Zhouchen Lin
arXiv:2609.37841v1 Announce Type: new
Abstract: Masked generative models offer parallel token prediction, but accurate parallel sampling must account for dependencies among tokens. When dependencies...
By Ryotaro Kawata, Satoshi Hayakawa, Taiji Suzuki
arXiv:2610. 13924v1 Announce Type: new Abstract: A parallel sampler can choose at random which coordinates to generate together.
By Tianyi Wu
arXiv:2608. 13520v1 Announce Type: cross Abstract: We study masking diffusion for discrete sampling and introduce a path-resolved measure of data geometry called the \emph{unmasking growth complexity} ({\textsf{UGC}\xspace}).
By Martin J. Wainwright
arXiv:2510. 04767v2 Announce Type: replace Abstract: While most autoregressive LLMs are constrained to one-by-one decoding, diffusion LLMs (dLLMs) have attracted growing interest for their potential to dramatically accelerate inference through parallel decoding.
By Wonjun Kang, Kevin Galim, Seunghyuk Oh, Minjae Lee, Yuchen Zeng, Shuibai Zhang, Coleman Hooper, Yuezhou Hu, Hyung Il Koo, Nam Ik Cho, Kangwook Lee
The paper introduces the Single-Increment Rejection Sampler (SIRS) to improve query efficiency in parallel masked diffusion models (MDMs). SIRS estimates the log acceptance probability using only one randomly sampled increment, then applies a Bernoulli factory to decide accept/reject outcomes. By Rao‑Blackwellizing each increment and bounding the KL‑divergence, SIRS reduces bias while maintaining parallel decoding performance.
By Shiyuan Zhang, Xuheng Li, Qiwei Di, Quanquan Gu
The paper compares the parallelism capabilities of three diffusion large language model paradigms—masked, uniform, and Gaussian diffusion. It proves that uniform and Gaussian diffusion can sample with a number of forward passes scaling with the dual total correlation of the distribution, potentially much less than the context length, whereas masked diffusion may require more passes. The study establishes a provable separation in parallelism, showing that masked diffusion’s critical windows are asymptotically narrower than those of the other two approaches.
By Sitan Chen, Liye Wang
The paper introduces random walks on graphs as a controllable sandbox for studying parallel sampling strategies in masked diffusion models (MDMs). It shows that common parallel unmasking methods, such as lowest‑entropy scoring, are not universally superior to random sampling and that performance depends on the graph’s conditional dependence structure. The authors also propose training‑free bisection samplers that are provably exact for random walks and demonstrate their effectiveness on both graph‑walk tasks and pretrained MDMs for language generation and reasoning benchmarks.
By Vansh Bansal, Cholyeon Cho, Syamantak Kumar, Sujay Sanghavi, Purnamrita Sarkar
arXiv:2608.20530v1 Announce Type: new
Abstract: Speculative decoding accelerates language-model inference by drafting future tokens that the target model verifies in parallel. A diffusion-style block...
By Matan Rusanovsky, Yoav Miron, Roy Uziel, Omer Belhasin, Ran Zilberstein, Maor Ashkenazi, Michael Elad
Masked diffusion models generate text by reversing a stochastic masking process, allowing many tokens to be revealed in parallel at the cost of errors due to independent token draws. The paper maps this trade‑off onto driven many‑particle systems, showing that for Markov data the process is a solvable random sequential adsorption model and that the error equals entropy production relative to the reversed masking. For many steps, the error corresponds to dissipation in a slowly driven system, with optimal schedules being geodesics of a thermodynamic metric derived from token mutual information, revealing how text correlations influence error concentration and output quality.
By Adrian Baule
arXiv:2606. 02955v3 Announce Type: replace-cross Abstract: Diffusion large language models promise parallel token generation, yet inference remains bottlenecked by deciding which masked tokens can be safely committed together.
By Siva Rajesh Kasa, Yasong Dai, Sumit Negi, Hongdong Li
arXiv:2610.34703v1 Announce Type: new
Abstract: Simulation-based inference often encodes observations $Y$ into a fixed-dimensional summary $S$, with the posterior conditioning only on $S$. While exac...
By Vansh Bansal, Pradip Sharma Poudel, James G. Scott