arXiv Machine Learning
Give the Drafter a Second Chance to Guess: Parallel Reconsideration for Speculative Decoding
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Sep 24
When Parallel Drafter Meets Parallel Speculative Decoding
The paper introduces DPara, a parallel speculative decoding framework that builds on DSpark-style parallel drafters. DPara eliminates the need for probabilistic guesses by precomputing draft representations for every acceptance boundary and using a lightweight autoregressive head to combine verification outcomes with these representations, enabling full parallelization of the backbone forward pass. Experiments on Qwen3-8B and Qwen3-14B across multiple benchmarks demonstrate average speedups of 3.21× and 3.52× over autoregressive decoding, outperforming existing serial and parallel speculative decoding methods.
arXiv AI
Aug 7
When Drafts Evolve: Speculative Decoding Meets Online Learning
arXiv:2603. 12617v2 Announce Type: replace-cross Abstract: Speculative decoding has emerged as a widely adopted paradigm for accelerating large language model inference, where a lightweight draft model rapidly generates candidate tokens that are then verified in parallel by a larger target model.
arXiv Computation and Language
Oct 1
DEdit: Iterative Draft Editing for Speculative Decoding
arXiv:2609.38510v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusi...
arXiv Machine Learning
Jun 3
TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding
arXiv:2606. 03819v1 Announce Type: new Abstract: One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation.
arXiv Machine Learning
Oct 8
Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds
The paper introduces the Expected Decoding Rounds (EDR) objective, a theoretical framework that models speculative decoding as a Markov reward process and directly optimizes the number of decoding rounds. EDR weights local rejection costs by state occupancies, eliminating auxiliary hyperparameters and enabling unbiased stochastic optimization via temporal‑difference gradients. Applying EDR to fine‑tune state‑of‑the‑art draft models (DSpark and DFly) improves mean accepted length and outperforms prior training objectives across nine benchmarks in math reasoning, code generation, and chat.
arXiv Machine Learning
Oct 21
DStill: Trajectory Consistency Distillation for Accelerated One-Step Drafting in Block Diffusion Speculative Decoding
arXiv:2610.27379v1 Announce Type: cross Abstract: Block-diffusion speculative decoding accelerates large language model inference by drafting multiple tokens in parallel. However, one-step drafting i...