Speculative decoding accelerates autoregressive generation by drafting a block of tokens that the target model verifies left-to-right, committing only the longest accepted prefix. Block (DLM-style) drafters predict the whole block in parallel, which is fast but trained with a full-block cross-entropy that supervises every position against the gold continuation -- even though inference discards every token after the first rejection.
arXiv:2607. 01893v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by drafting a block of tokens that the target model verifies left-to-right, committing only the longest accepted prefix.
arXiv:2608.30427v1 Announce Type: cross Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, pres...
arXiv:2608. 03447v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel.
arXiv:2609.36173v1 Announce Type: cross Abstract: Parallel speculative drafting generates multiple candidates in one backbone pass, but independent token selection can produce inconsistent continuati...
arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept seve...
Verification-Aware Training (VAT) is a plug‑in framework that improves speculative decoding for large language models by simulating verification during training and using the resulting accept/reject patterns as supervision. VAT adds a lightweight binary verification head to predict whether each draft token will survive sequential verification, and replaces the fixed per‑position weighting with a verification‑adaptive schedule that keeps full weight up to the first rejection point. When applied to EAGLE‑3 and DFlash on Qwen3‑4B, Qwen3‑8B, and LLaMA‑3.1‑8B, VAT increases average acceptance length by up to 11.4% and wall‑clock speedup by up to 8.7%, yielding consistent gains across math, code, and chat benchmarks.
arXiv:2609.38510v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusi...
The paper introduces the concepts of an information floor and a model gap to analyze block drafting in language models. By estimating these metrics across multiple domains and models, it finds that the all-parallel floor limits per-slot acceptance to 71% on Qwen3-4B, that a single realized token can eliminate most of this floor, and that current drafters still operate far above their floors, indicating significant room for improvement. These results highlight the distinct contributions of short-range conditioning versus proposal quality in block drafting.
The paper introduces Selective Supervision for Direct-OPD (S$^2$D-OPD), a refinement of Direct On-Policy Distillation that filters out states where the teacher’s policy change is minimal, as measured by the teacher‑reference Jensen‑Shannon divergence. By masking low‑divergence states and keeping only the top 10% of states per response, S$^2$D-OPD improves held‑out accuracy on AIME and HMMT benchmarks across multiple teacher‑student pairs without additional forward passes.
Block drafters generate multiple tokens in a single forward pass before earlier target tokens are produced, combining two loss components: missing within‑block path information and imperfect modeling of observable information. The study introduces an information floor—the minimum expected rejection for a given conditioning order—and defines the model gap as rejection above this floor. Across four domains and several models, the authors find that the all‑parallel floor limits per‑slot acceptance to 71% for Qwen3‑4B, that a single realized token can eliminate 86–100% of this floor, and that current drafters exhibit significant model gaps, accounting for 43–64% of DFlash rejection and 85–92% of DSpark’s oracle‑conditioned rejection.
Draft-OPD introduces an on‑policy distillation method for speculative draft models, addressing the mismatch between supervised fine‑tuning and inference by letting the target model supervise the drafter on draft‑induced states. The approach uses target‑assisted rollouts for stable continuations and replays drafting from error positions exposed during verification, enabling the drafter to learn from both accepted and rejected proposals. Experiments demonstrate that Draft‑OPD achieves more than five‑fold lossless acceleration across diverse tasks, outperforming prior draft models such as EAGLE‑3 and DFlash by 23 % and 13 % respectively.
The paper introduces AdaptiveSpec, a training‑free speculative decoding method that simultaneously adapts the per‑step verification rule and the draft‑tree shape using signals generated during decoding. It replaces the fixed token‑match rule with a margin‑based threshold and adjusts tree depth, width, and node count based on draft confidence and recent acceptance history, allowing the total draft count to vary. Experiments on SGLang show up to 56% throughput gains over EAGLE‑3 while maintaining 93% of lossless task accuracy on GSM8K, MATH‑500, and HumanEval across three models.