The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2610. 28355v1 Announce Type: new Abstract: In speculative decoding, a lightweight draft model proposes tokens that the target model verifies in parallel, speeding up generation without changing the target distribution.
arXiv:2608.30427v1 Announce Type: cross Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, pres...
The paper investigates how target model probability distributions can be reused during speculative decoding to improve token selection. It shows that exact reuse requires a distribution computed with the full next-round context, and that retaining this distribution can force the rejection of accepted tokens. Experiments across six target models demonstrate that reranking first-token branches with a separately computed distribution at the current context boosts expected acceptance, especially with small tree budgets, and that drawing the first token from this distribution before tree allocation further improves acceptance and decoding speed.
arXiv:2608.20530v1 Announce Type: new Abstract: Speculative decoding accelerates language-model inference by drafting future tokens that the target model verifies in parallel. A diffusion-style block...
arXiv:2609.24150v1 Announce Type: new Abstract: Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to generate multiple candidate tokens that are...
arXiv:2610.24993v2 Announce Type: replace Abstract: The throughput gains of speculative decoding depend on the trade-off between acceptance length and execution latency. We ask when additional accept...