arXiv Machine Learning By Marcello Bullo, Yanxiao Liu, \"Oyk\"u S{\i}la G\"uner, Arpan Mukherjee, Deniz G\"und\"uz

Rank-Aware Speculative Sampling for Diffusion Draft Trees

Read the original on arXiv Machine Learning →

The paper introduces Rank-Aware Speculative Sampling (RASS), a new verification rule for speculative draft trees in diffusion models. RASS orders draft candidates by proposal‑target mean displacement, samples a rank with weights optimized to minimize total variation, and then maximally couples the selected candidate with the target distribution, ensuring exact sampling. Experiments on Gaussian mixtures, FFHQ, CIFAR‑10, and Stable Diffusion 3.5 show that RASS reduces target‑model evaluation counts by up to 20% compared to Diffusion Greedy Rejection Sampling (D‑GRS).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Sep 17

Accelerating Diffusion Sampling via Speculative Draft Trees

The paper introduces "draft trees" to accelerate diffusion model sampling by allowing non‑linear lookahead drafts, thereby increasing acceptance rates per expensive target evaluation. It connects speculative sampling to relative entropy coding, adopts greedy rejection sampling as the draft‑target coupling, and demonstrates up to 8.3% speed‑up over reflection coupling baselines in experiments.

By Marcello Bullo, Yanxiao Liu, \"Oyk\"u S{\i}la G\"uner, Arpan Mukherjee, Deniz G\"und\"uz
arXiv Computation and Language
Oct 22

Understanding Target Probability Reuse in Speculative Decoding

The paper investigates how target model probability distributions can be reused during speculative decoding to improve token selection. It shows that exact reuse requires a distribution computed with the full next-round context, and that retaining this distribution can force the rejection of accepted tokens. Experiments across six target models demonstrate that reranking first-token branches with a separately computed distribution at the current context boosts expected acceptance, especially with small tree budgets, and that drawing the first token from this distribution before tree allocation further improves acceptance and decoding speed.

By Jungseob Lee, Giyoon Jang, Chailin Nam, Jiwoo Kum, Sihyeon Lee, Kyojun Choo, Jihun Song, Eunseo Song, Chanjun Park
arXiv Machine Learning
Oct 28

Multiple Importance Sampling for Guided Speculative Inference

The paper extends Guided Speculative Inference (GSI) for test‑time scaling of large language models by allowing an ensemble of smaller LLMs to propose draft reasoning steps, which are then verified by a larger base model and a reward model. Using multiple importance sampling (MIS), the authors introduce weighting schemes to combine samples from diverse proposal distributions, prove new bounds on KL divergence and expected reward loss, and show that the balance heuristic is optimal. Experiments demonstrate that combining several diverse LLMs, while keeping the total number of samples fixed, achieves competitive downstream performance and is mathematically robust when a single draft model’s support does not cover the target distribution, a scenario where standard GSI may fail.

By Joseph Rowan, Ashish J. Khisti