arXiv:2609. 04010v1 Announce Type: new Abstract: Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation.
arXiv:2606. 02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding.
arXiv:2610.28559v1 Announce Type: cross Abstract: Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet ef...
arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.
arXiv:2601. 07568v3 Announce Type: replace-cross Abstract: Diffusion large language models (dLLMs) offer capabilities beyond those of autoregressive (AR) LLMs, such as parallel decoding and random-order generation.
arXiv:2608.22646v1 Announce Type: new Abstract: Diffusion language models can generate many tokens in parallel, but they still require repeated denoising steps during inference. This makes generation...
Zarya is a hybrid language model that jointly trains an autoregressive objective and a masked-diffusion objective within a single architecture. It structures training data into variable-size slots and uses a curriculum that gradually increases slot granularity, allowing a smooth transition from fine-grained AR learning to coarse-grained diffusion learning. At inference, Zarya offers two decoding modes—MDM sampling with first-hitting denoising and slotted speculative decoding that interleaves diffusion-based selection with autoregressive infilling—while fully decoupling training and inference regimes and supporting extensive configurability.
arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.
The paper introduces DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores intra-block dependencies in speculative decoding by evaluating multiple candidate tokens for contextual compatibility across adjacent draft positions. DCOCO maintains parallel drafting while enabling contextually informed token selection at low additional cost. Across multiple verifier backbones and benchmarks, DCOCO achieves the highest average throughput, delivering speedups of up to 6.6× over vanilla decoding and outperforming competing methods by 6–23% under concurrent serving loads.
On-Policy Residual Learning for Diffusion Language Models (ORL) introduces a framework that learns residual context directly from model-generated trajectories, using an entropy-contraction objective to sharpen predictions. The approach co-designs algorithmic and system components—such as asynchronous residual capture, block-diffusion attention, and packed compute-balanced training—to enable practical scaling. Experiments on a 4B DLM converted from Qwen3-4B show up to 6.2× faster tokens per forward and 6.9× higher end-to-end throughput, while also improving speed-quality trade-offs over autoregressive models and transferring effectively to other DLMs like LLaDA-2.1-mini and DiffusionGemma.
arXiv:2509. 18085v4 Announce Type: replace-cross Abstract: Diffusion LLMs (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs (AR-LLMs) with the potential to operate at significantly higher token-generation rates.
The paper introduces dSpark, a schedule distillation framework that learns when to commit tokens in diffusion large language models to accelerate generation. By constructing unmasking schedules that prioritize revealing tokens which make other positions easier to predict, dSpark reduces uncertainty earlier in decoding. Experiments on mathematical reasoning and code generation tasks show significant speedups—10.12× on LLaDA‑8B‑Instruct and 5.12× on Dream‑7B‑Instruct—while preserving superior task performance.