arXiv Machine Learning
Acceptance-Aware Draft Model Training for Speculative Decoding
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 2
LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding
arXiv:2602. 23881v2 Announce Type: replace Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model.
arXiv AI
Jun 9
Variational Speculative Decoding: Rethinking Draft Training from Token Likelihood to Sequence Acceptance
arXiv:2602. 05774v4 Announce Type: replace-cross Abstract: Speculative decoding accelerates inference for (M)LLMs, yet a training-decoding discrepancy persists: while existing methods optimize single greedy trajectories, decoding involves verifying and ranking multiple sampled draft paths.
arXiv Machine Learning
Jun 30
When Is a Draft Accepted? A Theory of Acceptance in Speculative Decoding
arXiv:2606. 30265v1 Announce Type: new Abstract: Speculative decoding accelerates language model inference by using a fast drafter to propose candidate tokens that are then verified by a larger target model.
arXiv AI
Aug 11
LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization
arXiv:2608. 08721v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round.
Hugging Face Trending Papers
Aug 25
ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping
The efficiency of Large Language Model (LLM) serving is fundamentally limited by the sequential nature of autoregressive decoding. Speculative Decoding (SD) mitigates this by using a lightweight draft...
arXiv AI
Aug 26
ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping
ResiSpec is a framework that improves speculative decoding for large language models by reshaping the residual distribution during verification. It addresses the problem of residual drift, where rejected candidates cause the target distribution to diverge from the draft model’s predictions, rendering later candidates ineffective. By aligning the verification process with the draft model’s high‑confidence regions, ResiSpec prevents candidate obsolescence and achieves up to 1.92× speedup over existing multi‑candidate methods.