arXiv AI
Jun 2
IDLM: Inverse-distilled Diffusion Language Models
arXiv:2602. 19066v2 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) have recently achieved strong results in text generation.
DLM-One: Diffusion Language Models for One-Step Sequence Generation
The paper introduces DLM-One, a score‑distillation framework that enables one‑step sequence generation with continuous diffusion language models (DLMs). By aligning a student model’s outputs with a pretrained teacher DLM’s score function in the forward‑diffused noisy space, DLM-One removes the need for iterative refinement. Experiments across various DLM architectures show up to ~2000× speedup in sampling steps and ~500× in wall‑clock time while retaining competitive performance, and the authors propose an adversarially‑regularized two‑stage training scheme to mitigate student degeneration.
Related stories
arXiv Machine Learning
Jul 7
Few-Step Diffusion Language Models via Trajectory Self-Distillation
arXiv:2602. 12262v4 Announce Type: replace-cross Abstract: Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding.
arXiv Machine Learning
Aug 4
Just on Time: Token-Level Early Stopping for Diffusion Language Models
arXiv:2602. 11133v2 Announce Type: replace Abstract: Diffusion language models generate text through iterative refinement, a process that is often computationally inefficient because many tokens reach stability long before the final denoising step.
arXiv Machine Learning
Jun 5
A Survey on Diffusion Language Models
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.
arXiv AI
Jun 19
Diffusion Language Models: An Experimental Analysis
arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.
arXiv AI
Jun 16
Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens
arXiv:2606. 16847v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality.
arXiv Machine Learning
1d ago
Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One
arXiv:2609.04531v1 Announce Type: new Abstract: Language generation is almost universally treated as a sequential process: autoregressive models emit one token at a time, while diffusion language mod...
arXiv Machine Learning
Jul 23
Multi-Mask Diffusion Language Models for Few-Step Generation
arXiv:2607. 19686v1 Announce Type: cross Abstract: Masked diffusion models (MDMs) are a promising family of language generators, but achieving high-quality few-step generation remains challenging.
arXiv Machine Learning
Jun 16
Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design
arXiv:2606. 15327v1 Announce Type: new Abstract: Diffusion Language Models (DLMs) have demonstrated strong scaling capacity as alternatives to autoregressive language models.
arXiv AI
Aug 24
Efficient Self-Evaluation for Diffusion Language Models via Sequence Regeneration
arXiv:2603.02760v2 Announce Type: replace-cross Abstract: Diffusion large language models (dLLMs) have recently attracted significant attention for their ability to enhance diversity, controllability...
arXiv Computation and Language
Aug 28
Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference
The paper introduces Dependency-Aware Revocable Decoding (DARD), a training‑free framework for diffusion large language models that separates tokens into masked, candidate, and unmasked states. DARD verifies candidate tokens using a selective context that excludes less reliable tokens and adaptively regulates their influence on subsequent decoding. Experiments on 12 textual and multimodal benchmarks across three open‑source dLLMs show that DARD improves the speed‑quality Pareto frontier, achieving a 2.71× speedup and a 4.35‑point CIDEr gain over Saber on Flickr30K.
arXiv AI
Jun 30
ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation
arXiv:2606. 29869v1 Announce Type: cross Abstract: Knowledge distillation (KD) is a key technique for compressing Large Language Models (LLMs), yet methods relying on a single KL objective often fail to balance primary distribution fitting with long-tail probability modeling, limiting both generation quality and generalization.