Trajectory-Level Speculative Decoding for Diffusion Language Models

The paper introduces a trajectory-level speculative decoding framework for diffusion-based language models (dLLMs), addressing the limitation of existing strategies that revert to single-token generation when confidence is low. By constructing draft denoising trajectories through confidence-stratified tree exploration and verifying them with blockwise parallel evaluation and bidirectional attention masking, the method also incorporates inter-block speculation to exploit the models’ bidirectional structure. Experiments show a 30–40% reduction in denoising iterations, a token-per-step increase from 2.6 to 4.3, and a 7–14× speedup over vanilla dLLMs while maintaining accuracy within 1% on reasoning and code benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• diffusion
• efficiency
• benchmarks