arXiv Computation and Language
Recovering Off-Policy Supervision for Speculative Decoding
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jul 10
A Practical Investigation of Training-free Relaxed Speculative Decoding
arXiv:2607. 08690v1 Announce Type: cross Abstract: Speculative decoding accelerates sampling from an autoregressive LLM by using a faster auxiliary model to draft tokens which are then verified in parallel by the LLM.
arXiv Computation and Language
3d ago
DEdit: Iterative Draft Editing for Speculative Decoding
arXiv:2609.38510v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusi...
Hugging Face Trending Papers
Jun 4
AdaPLD: Adaptive Retrieval and Reuse for Efficient Model-Free Speculative Decoding
Speculative decoding accelerates generation by verifying multiple drafted tokens in a single target-model forward pass, reducing sequential decoding iterations. Model-free variants avoid auxiliary draft models by reusing text and model states already available during generation, but their speedup depends on the reliability of the constructed drafts.
Hugging Face Trending Papers
Jul 9
A Practical Investigation of Training-free Relaxed Speculative Decoding
Speculative decoding accelerates sampling from an autoregressive LLM by using a faster auxiliary model to draft tokens which are then verified in parallel by the LLM. Standard speculative decoding is lossless: its rejection and resampling steps exactly preserve the LLM's sampling distribution.
arXiv AI
Sep 15
Carryover Drafting: Recycling Rejected States for Speculative Decoding
arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept seve...
arXiv Computation and Language
Sep 1
ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding
arXiv:2608.29748v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...