Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path.
arXiv:2608. 13524v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel.
By Tianyi Li, Yaxin Luo, Xinyi Shang, Zhiqiang Shen
arXiv:2608.20375v1 Announce Type: new
Abstract: Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree b...
By Xuming Ye, Zeming Ma, Runjie Yu, Yuan Liu, Tianle Li, Shuhan Bai, Jian Zhou, Fei Wu
arXiv:2609.22098v1 Announce Type: new
Abstract: Speculative decoding accelerates language-model inference by letting a cheap drafter propose tokens that the target model verifies in parallel. Recent...
By Huapeng Zhou, Huayu Wang, Xinyu Wang
arXiv:2606. 00487v1 Announce Type: new Abstract: Using a diffusion model for parallel drafting is a promising approach for speculative decoding.
By Zhuoyu Wang, Junnan Huang, Xinyu Chen
Bastion is a budget‑aware speculative decoding framework that uses tree‑structured block diffusion drafting to improve the efficiency of language model inference. It dynamically builds query‑dependent trees, balancing draft quality with hardware constraints, and incorporates an acceptance surrogate, an online latency estimator, and an adaptive best‑first expansion to maximize speed without sacrificing model distribution. The method achieves up to a 6.61× speedup over standard autoregressive decoding and outperforms state‑of‑the‑art block‑diffusion baselines by 39% across various benchmarks and GPU architectures.
By Soowon Oh, Nam Cao, Yujin Kim, Hojung Jung, Huzama Ahmad, Sangmin Bae, Se-Young Yun