arXiv:2606. 03819v1 Announce Type: new Abstract: One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation.
arXiv:2608.20375v1 Announce Type: new Abstract: Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree b...
TreeGraft introduces a multi-drafter framework that combines drafters of varying costs to build a shared draft tree for tree-based speculative decoding. The stronger drafter rescues and rescoring candidates from the weaker drafter, while a lightweight scheduler decides when to invoke the stronger drafter to manage cost. Experiments on 10 model pairs and 6 benchmarks show TreeGraft improves over the best single-drafter strategy by an average of 15.1% and up to 26.6%.
Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path.
arXiv:2606. 00487v1 Announce Type: new Abstract: Using a diffusion model for parallel drafting is a promising approach for speculative decoding.
The paper introduces Tree Exit Verification (TEV), an exact, level‑parallel speculative decoding method that uses a single exit‑node decision and a bonus‑token decision to verify draft continuations. By deriving a canonical exit law, the authors show that TEV achieves the same acceptance as optimal verifiers while reducing verifier‑stage latency by 15%. They also propose Exit‑Guided Draft‑Tree Training (ExitTrain), which improves draft trees and increases average output‑block length by 13%, leading to a 14% end‑to‑end speedup over DDTree across dialogue, code, and mathematical reasoning tasks.
arXiv:2607. 22634v1 Announce Type: new Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs, generating tokens in parallel.
arXiv:2609.22098v1 Announce Type: new Abstract: Speculative decoding accelerates language-model inference by letting a cheap drafter propose tokens that the target model verifies in parallel. Recent...
arXiv:2610.07822v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target...
arXiv:2609.39972v1 Announce Type: new Abstract: Speculative decoding accelerates language model inference by verifying multiple draft tokens in a single target-model pass. Recent parallel drafters ha...
arXiv:2608. 13524v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel.
CAST (Cost‑Aware Speculative Trees) is a method that improves speculative decoding for large language models by packing multiple drafted token candidates into a tree and verifying the entire tree in a single target‑model pass, rather than only the top‑scoring chain. The tree width is adaptively chosen based on a latency measurement, ensuring that each added candidate’s expected gain outweighs its verification cost. Experiments across five domains, three GPU generations, and two model families show that CAST can be up to 43 % faster than the standard chain, while preserving the target model’s output distribution under both greedy and sampled decoding.