TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding
arXiv:2606. 00487v1 Announce Type: new Abstract: Using a diffusion model for parallel drafting is a promising approach for speculative decoding.
The paper introduces Rank-Aware Speculative Sampling (RASS), a new verification rule for speculative draft trees in diffusion models. RASS orders draft candidates by proposal‑target mean displacement, samples a rank with weights optimized to minimize total variation, and then maximally couples the selected candidate with the target distribution, ensuring exact sampling. Experiments on Gaussian mixtures, FFHQ, CIFAR‑10, and Stable Diffusion 3.5 show that RASS reduces target‑model evaluation counts by up to 20% compared to Diffusion Greedy Rejection Sampling (D‑GRS).
arXiv:2606. 00487v1 Announce Type: new Abstract: Using a diffusion model for parallel drafting is a promising approach for speculative decoding.
The paper introduces "draft trees" to accelerate diffusion model sampling by allowing non‑linear lookahead drafts, thereby increasing acceptance rates per expensive target evaluation. It connects speculative sampling to relative entropy coding, adopts greedy rejection sampling as the draft‑target coupling, and demonstrates up to 8.3% speed‑up over reflection coupling baselines in experiments.
arXiv:2607. 22634v1 Announce Type: new Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs, generating tokens in parallel.
The paper investigates how target model probability distributions can be reused during speculative decoding to improve token selection. It shows that exact reuse requires a distribution computed with the full next-round context, and that retaining this distribution can force the rejection of accepted tokens. Experiments across six target models demonstrate that reranking first-token branches with a separately computed distribution at the current context boosts expected acceptance, especially with small tree budgets, and that drawing the first token from this distribution before tree allocation further improves acceptance and decoding speed.
Speculative decoding speeds up LLM inference by using a draft model to generate tokens, with an acceptance-rejection scheme that ensures that the output matches the target distribution. Adapting this to continuous diffusions is difficult because speculative sampling requires drawing from a residual distribution.
The paper extends Guided Speculative Inference (GSI) for test‑time scaling of large language models by allowing an ensemble of smaller LLMs to propose draft reasoning steps, which are then verified by a larger base model and a reward model. Using multiple importance sampling (MIS), the authors introduce weighting schemes to combine samples from diverse proposal distributions, prove new bounds on KL divergence and expected reward loss, and show that the balance heuristic is optimal. Experiments demonstrate that combining several diverse LLMs, while keeping the total number of samples fixed, achieves competitive downstream performance and is mathematically robust when a single draft model’s support does not cover the target distribution, a scenario where standard GSI may fail.
Bastion is a budget‑aware speculative decoding framework that uses tree‑structured block diffusion drafting to improve the efficiency of language model inference. It dynamically builds query‑dependent trees, balancing draft quality with hardware constraints, and incorporates an acceptance surrogate, an online latency estimator, and an adaptive best‑first expansion to maximize speed without sacrificing model distribution. The method achieves up to a 6.61× speedup over standard autoregressive decoding and outperforms state‑of‑the‑art block‑diffusion baselines by 39% across various benchmarks and GPU architectures.
arXiv:2608.20375v1 Announce Type: new Abstract: Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree b...
DForkTree is a training‑free method that improves speculative decoding by re‑anchoring each branch of a draft tree on its own tokens. It adjusts drafter scores using pointwise mutual information from target‑generated n‑grams, weighting these adjustments more heavily after a fork, and employs a layer‑wise GPU search to recover the exact optimal tree. On Qwen3.5‑4B and 9B models, DForkTree outperforms all baselines, achieving up to 41.5% better results than DFlash, 15.1% better than DDTree, and average speedups of up to 4.75× over autoregressive decoding.
TreeGraft introduces a multi-drafter framework that combines drafters of varying costs to build a shared draft tree for tree-based speculative decoding. The stronger drafter rescues and rescoring candidates from the weaker drafter, while a lightweight scheduler decides when to invoke the stronger drafter to manage cost. Experiments on 10 model pairs and 6 benchmarks show TreeGraft improves over the best single-drafter strategy by an average of 15.1% and up to 26.6%.
arXiv:2606. 04446v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive large language model inference by drafting multiple tokens and verifying them in a single target-model forward pass.
The efficiency of Large Language Model (LLM) serving is fundamentally limited by the sequential nature of autoregressive decoding. Speculative Decoding (SD) mitigates this by using a lightweight draft...