arXiv:2610. 28355v1 Announce Type: new Abstract: In speculative decoding, a lightweight draft model proposes tokens that the target model verifies in parallel, speeding up generation without changing the target distribution.
arXiv:2608.30427v1 Announce Type: cross Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, pres...
The paper investigates how target model probability distributions can be reused during speculative decoding to improve token selection. It shows that exact reuse requires a distribution computed with the full next-round context, and that retaining this distribution can force the rejection of accepted tokens. Experiments across six target models demonstrate that reranking first-token branches with a separately computed distribution at the current context boosts expected acceptance, especially with small tree budgets, and that drawing the first token from this distribution before tree allocation further improves acceptance and decoding speed.
arXiv:2608.20530v1 Announce Type: new Abstract: Speculative decoding accelerates language-model inference by drafting future tokens that the target model verifies in parallel. A diffusion-style block...
arXiv:2609.24150v1 Announce Type: new Abstract: Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to generate multiple candidate tokens that are...
arXiv:2610.24993v2 Announce Type: replace Abstract: The throughput gains of speculative decoding depend on the trade-off between acceptance length and execution latency. We ask when additional accept...
arXiv:2610.07822v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target...
arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept seve...
arXiv:2608. 03447v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel.
arXiv:2608.02438v2 Announce Type: replace Abstract: Block-diffusion drafters like dFlash generate an entire block of draft tokens in a single forward pass, drastically reducing the overhead of multip...
arXiv:2607. 21535v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by having a cheap draft propose tokens that a target verifies in parallel.
The paper explores a new lossy strategy for speculative decoding called verifier skipping, which allows certain draft prefixes to be committed without invoking the verifier. It investigates which confidence signals—raw token confidence, learned marginal survival, or learned conditional survival—best schedule these skips, finding that raw confidence often performs best despite higher positionwise AUROC for learned signals. Experiments on HumanEval with DiffuCoder-7B-Instruct and Qwen3-32B show that all three signals can reduce verifier calls by 9.6% to 13.5% while maintaining the same pass@1 as Strict SDD.