arXiv:2607. 24507v1 Announce Type: cross Abstract: Existing methods mainly adapt pretrained autoregressive (AR) language models to masked diffusion, whereas we directly adapt them to uniform-noise diffusion, where every token remains editable during sampling.
CoDR (Confidence Drift Remasking) is a training‑free, sampler‑agnostic refinement pass for masked diffusion language models that detects and corrects early token commitments that become unsupported as context grows denser. It estimates confidence drift for all committed positions using only a few forward passes and remasks only the tokens whose confidence has dropped, thereby improving accuracy across multiple backbones, tasks, and samplers with modest computational overhead.
arXiv:2606. 02241v1 Announce Type: new Abstract: Is the uniform-state diffusion framework a more powerful paradigm for discrete diffusion?
arXiv:2605. 13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling.
The paper introduces a lightweight single‑layer sampler that allows masked diffusion language models to approximate joint sampling of multiple tokens in a single full‑model forward pass. By training the sampler to mimic exact joint sampling from a frozen diffusion model, the authors enable parallel unmasking of tokens while maintaining a close match to the true joint distribution. Experiments on Dream‑7B and Llada‑7B models show that unmasking four tokens per denoising step yields a MAUVE score of 0.87, a substantial improvement over the marginal baseline of 0.31.
arXiv:2607. 24763v1 Announce Type: new Abstract: Masked diffusion language models (MDLMs) are advancing rapidly, yet the evaluation standards needed to reliably interpret their progress have not kept pace.
arXiv:2610.34310v1 Announce Type: cross Abstract: Insertion-based masked diffusion models enable flexible-length, parallel generation by interleaving mask insertion with token unmasking. However, bec...
The paper revisits the continuous diffusion language model Plaid and introduces RePlaid, aligning its architecture with modern discrete diffusion models. RePlaid achieves a compute gap of only 20× compared to autoregressive models, surpasses Duo with fewer parameters, and outperforms MDLM in over‑trained settings. On OpenWebText, RePlaid sets a new state‑of‑the‑art continuous diffusion perplexity of 22.1 and demonstrates superior generation quality, while theoretical analysis links likelihood‑based training to linear cross‑entropy over time and structured embedding geometries.
arXiv:2610.17266v1 Announce Type: new Abstract: Diffusion language models are a promising and more efficient alternative to autoregressive models. However, most RL post-training techniques are not ea...
The paper introduces Random Remask-Denoise (R$^2$D) inference, a test‑time, finetuning‑free method for masked diffusion models that randomly re‑introduces masks and denoises to form a Markov chain on clean sequences. It also proposes Trajectory Correction Training (TCT), which trains the denoiser by exposing it to its own errors during generation. Experiments show that R$^2$D with TCT achieves near‑log‑linear scaling, achieving over 97% accuracy on hard Sudoku puzzles, over 99% on 3‑SAT, and surpasses iterative correction baselines on language tasks such as HumanEval+.
arXiv:2609. 04010v1 Announce Type: new Abstract: Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation.
The paper introduces the Single-Increment Rejection Sampler (SIRS) to improve query efficiency in parallel masked diffusion models (MDMs). SIRS estimates the log acceptance probability using only one randomly sampled increment, then applies a Bernoulli factory to decide accept/reject outcomes. By Rao‑Blackwellizing each increment and bounding the KL‑divergence, SIRS reduces bias while maintaining parallel decoding performance.