The study compares continuous diffusion language models (CDLMs) and speculative drafters by training them from scratch on unconditional OpenWebText generation at lengths 16, 128, and 1024. Using LangFlow for CDLMs and adapted DSpark Markov and recurrent-head architectures for standalone generation, the authors evaluate generation perplexity (GenPPL) and MAUVE similarity. Results show LangFlow outperforms both DSpark variants at longer lengths (128 and 1024), while DSpark-RNN leads at length 16; recurrent memory and teacher probability matching improve metrics but the longer‑sequence gap remains, highlighting iterative full‑sequence refinement as a key advantage.
arXiv:2607. 19223v1 Announce Type: new Abstract: Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference.
Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference. Recent work such as DFlash further boosts drafting efficiency by leveraging diffusion drafters, whose parallel denoising mechanism enables draft generation in a single forward pass.
arXiv:2606. 11552v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs due to inherently sequential token generation.
Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs due to inherently sequential token generation. Speculative decoding addresses this bottleneck by employing a lightweight draft model to propose multiple future tokens that are subsequently verified in parallel by a larger target model.
The paper introduces T2D, a framework that prunes and adapts a target large language model into a draft model optimized for both inference latency and alignment with the target model’s outputs. By formulating draft model design as an optimization problem, T2D achieves up to 92% pruning while maintaining high speculative decoding performance across multiple target model families and domains. The resulting draft models can also be integrated with other speculative decoding techniques to yield additional gains.
The paper introduces DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores intra-block dependencies in speculative decoding by evaluating multiple candidate tokens for contextual compatibility across adjacent draft positions. DCOCO maintains parallel drafting while enabling contextually informed token selection at low additional cost. Across multiple verifier backbones and benchmarks, DCOCO achieves the highest average throughput, delivering speedups of up to 6.6× over vanilla decoding and outperforming competing methods by 6–23% under concurrent serving loads.
arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.
NgramRL is a new method for accelerating reinforcement‑learning post‑training of large language models by using a per‑prompt n‑gram drafter built from recent rollout history, eliminating the need for auxiliary models or persistent GPU state. It incorporates an adaptive fan‑out prefetcher that predicts verification outcomes to hide CPU cache lookup and scheduling delays, and a causal budget allocator that focuses prefetches on high‑probability outcomes. Experiments on two model families and three reasoning/structured‑generation workloads show that NgramRL achieves 3.60–7.77 token savings per target‑model forward in the warm‑cache regime, up to 1.99× faster than SuffixDecoding and 7.0× faster than prompt‑lookup decoding, with the prefetcher improving throughput by 1.38× on Spider.
The paper introduces PILL, a new infilling technique for diffusion language models that eliminates the need for a preset initial length and reduces inference overhead. PILL uses probing-based length-free decoding, cutting down on extra forward passes and speeding up generation. Experiments across five diffusion models and eight benchmarks show PILL outperforms the strongest baseline with higher pass rates and BLEU-2 scores while running 1.82× faster.
arXiv:2606. 04446v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive large language model inference by drafting multiple tokens and verifying them in a single target-model forward pass.
arXiv:2608.29748v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...
arXiv:2606. 19475v1 Announce Type: new Abstract: Large Language Models (LLMs) have revolutionized language modeling through autoregressive generation, enabling strong performance across a wide range of tasks.