arXiv:2606. 00487v1 Announce Type: new Abstract: Using a diffusion model for parallel drafting is a promising approach for speculative decoding.
By Zhuoyu Wang, Junnan Huang, Xinyu Chen
The paper introduces "draft trees" to accelerate diffusion model sampling by allowing non‑linear lookahead drafts, thereby increasing acceptance rates per expensive target evaluation. It connects speculative sampling to relative entropy coding, adopts greedy rejection sampling as the draft‑target coupling, and demonstrates up to 8.3% speed‑up over reflection coupling baselines in experiments.
By Marcello Bullo, Yanxiao Liu, \"Oyk\"u S{\i}la G\"uner, Arpan Mukherjee, Deniz G\"und\"uz
arXiv:2607. 22634v1 Announce Type: new Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs, generating tokens in parallel.
By Zheng Wang, Zhifan Ye, Qi Cheng, Yonggan Fu, Ziyan Wang, Feng Zhu, Haozhe Zhao, Jan Kautz, Pavlo Molchanov, Humphrey Shi, Minjia Zhang
The paper investigates how target model probability distributions can be reused during speculative decoding to improve token selection. It shows that exact reuse requires a distribution computed with the full next-round context, and that retaining this distribution can force the rejection of accepted tokens. Experiments across six target models demonstrate that reranking first-token branches with a separately computed distribution at the current context boosts expected acceptance, especially with small tree budgets, and that drawing the first token from this distribution before tree allocation further improves acceptance and decoding speed.
By Jungseob Lee, Giyoon Jang, Chailin Nam, Jiwoo Kum, Sihyeon Lee, Kyojun Choo, Jihun Song, Eunseo Song, Chanjun Park
Speculative decoding speeds up LLM inference by using a draft model to generate tokens, with an acceptance-rejection scheme that ensures that the output matches the target distribution. Adapting this to continuous diffusions is difficult because speculative sampling requires drawing from a residual distribution.
The paper extends Guided Speculative Inference (GSI) for test‑time scaling of large language models by allowing an ensemble of smaller LLMs to propose draft reasoning steps, which are then verified by a larger base model and a reward model. Using multiple importance sampling (MIS), the authors introduce weighting schemes to combine samples from diverse proposal distributions, prove new bounds on KL divergence and expected reward loss, and show that the balance heuristic is optimal. Experiments demonstrate that combining several diverse LLMs, while keeping the total number of samples fixed, achieves competitive downstream performance and is mathematically robust when a single draft model’s support does not cover the target distribution, a scenario where standard GSI may fail.
By Joseph Rowan, Ashish J. Khisti