arXiv AI By Zhicheng Zheng, Cheng Zhang, Chenkai Tang, Tianxiang Zhao

DForkTree: Accelerating Speculative Decoding with Fork-Aware Block Diffusion Draft Trees

Read the original on arXiv AI →

DForkTree is a training‑free method that improves speculative decoding by re‑anchoring each branch of a draft tree on its own tokens. It adjusts drafter scores using pointwise mutual information from target‑generated n‑grams, weighting these adjustments more heavily after a fork, and employs a layer‑wise GPU search to recover the exact optimal tree. On Qwen3.5‑4B and 9B models, DForkTree outperforms all baselines, achieving up to 41.5% better results than DFlash, 15.1% better than DDTree, and average speedups of up to 4.75× over autoregressive decoding.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Aug 13

DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees

Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path.

arXiv Machine Learning
Oct 28

Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting

Bastion is a budget‑aware speculative decoding framework that uses tree‑structured block diffusion drafting to improve the efficiency of language model inference. It dynamically builds query‑dependent trees, balancing draft quality with hardware constraints, and incorporates an acceptance surrogate, an online latency estimator, and an adaptive best‑first expansion to maximize speed without sacrificing model distribution. The method achieves up to a 6.61× speedup over standard autoregressive decoding and outperforms state‑of‑the‑art block‑diffusion baselines by 39% across various benchmarks and GPU architectures.

By Soowon Oh, Nam Cao, Yujin Kim, Hojung Jung, Huzama Ahmad, Sangmin Bae, Se-Young Yun