arXiv:2607. 24507v1 Announce Type: cross Abstract: Existing methods mainly adapt pretrained autoregressive (AR) language models to masked diffusion, whereas we directly adapt them to uniform-noise diffusion, where every token remains editable during sampling.
The paper introduces Corrective Diffusion Language Models (CDLMs), which enhance masked diffusion language models by explicitly supervising visible corrupted tokens to enable confidence-guided refinement of incorrect tokens. Experiments show that this correction-oriented post‑training improves iterative refinement and parallel decoding across models ranging from 0.5B to 8B, outperforming standard masked‑diffusion fine‑tuning on benchmarks such as Code Revision, HumanEval(+), and MATH. The authors also provide a new Code Revision Benchmark and release code and data for reproducibility.
arXiv:2606. 20560v1 Announce Type: cross Abstract: LLM reasoning transparency is a critical affordance for understanding model decisions, mitigating misuse and misalignment, and debugging surprising model behaviors.
arXiv:2608. 11235v1 Announce Type: new Abstract: Diffusion language models (DLMs) update many tokens in parallel, yet practical decoders often use a fixed denoising horizon.
arXiv:2605. 19262v2 Announce Type: replace Abstract: Masked diffusion language models (MDLMs) are emerging as a compelling new paradigm for text generation, but their training-time security remains largely unexplored.
arXiv:2604. 18738v3 Announce Type: replace Abstract: Diffusion language models (dLLMs) generate text through iterative denoising, filling multiple masked positions at each step.
arXiv:2609.37974v1 Announce Type: cross Abstract: Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The m...
arXiv:2607. 27386v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refinement.
The paper introduces Spherical Brownian Diffusion, a new continuous diffusion language model that uses Brownian motion on the unit sphere as its forward process. It provides a principled heat semigroup, an additive dimension‑free noise clock, and an exact ancestral bridge, while a clean‑token posterior learned with cross‑entropy enables multiple samplers. Experiments on Sudoku and GSM8K show that the choice of corruption process has little effect, whereas sampler selection can significantly impact performance, with memory and token‑selection effects differing across tasks.
CoDR (Confidence Drift Remasking) is a training‑free, sampler‑agnostic refinement pass for masked diffusion language models that detects and corrects early token commitments that become unsupported as context grows denser. It estimates confidence drift for all committed positions using only a few forward passes and remasks only the tokens whose confidence has dropped, thereby improving accuracy across multiple backbones, tasks, and samplers with modest computational overhead.
arXiv:2606. 08810v1 Announce Type: cross Abstract: Gaussian-corrupted sentence embeddings have no direct linguistic interpretation, yet continuous diffusion language models can generate fluent text from them.
The paper investigates two key design levers for Continuous Diffusion Language Models (CDLMs): self‑conditioning and embedding geometry. It shows that self‑conditioning mitigates a train/inference mismatch and can be implemented efficiently via a two‑level interpolant called DoubleCat, which significantly boosts TinyGSM accuracy. By adding discrete reveals to DoubleCat, the authors create SemiCat, achieving discrete‑diffusion‑level performance and outperforming discrete diffusion on several likelihood‑based benchmarks at 1.7B parameters. The study also finds that cross‑entropy training drives embeddings toward a geometry with decoding error rates comparable to one‑hot representations, and briefly explores additional options such as hybrid corruption processes and asynchronous noise schedules.