arXiv Computation and Language August 28, 2026 By Jiaming Fan, Daming Cao, Canchen Huang, Jiale Fu, Jin Zhang, Junjie Gao, Kai Yang, Xiangzhong Luo, Xu Yang

TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding

Read the original on arXiv Computation and Language →

TreeGraft introduces a multi-drafter framework that combines drafters of varying costs to build a shared draft tree for tree-based speculative decoding. The stronger drafter rescues and rescoring candidates from the weaker drafter, while a lightweight scheduler decides when to invoke the stronger drafter to manage cost. Experiments on 10 model pairs and 6 benchmarks show TreeGraft improves over the best single-drafter strategy by an average of 15.1% and up to 26.6%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

Aug 24

GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring

arXiv:2608.20375v1 Announce Type: new Abstract: Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree b...

By Xuming Ye, Zeming Ma, Runjie Yu, Yuan Liu, Tianle Li, Shuhan Bai, Jian Zhou, Fei Wu

llms diffusion efficiency

arXiv AI

Oct 9

Where Draft Trees Lose Target Mass: Exit-Guided Speculative Decoding

The paper introduces Tree Exit Verification (TEV), an exact, level‑parallel speculative decoding method that uses a single exit‑node decision and a bonus‑token decision to verify draft continuations. By deriving a canonical exit law, the authors show that TEV achieves the same acceptance as optimal verifiers while reducing verifier‑stage latency by 15%. They also propose Exit‑Guided Draft‑Tree Training (ExitTrain), which improves draft trees and increases average output‑block length by 13%, leading to a 14% end‑to‑end speedup over DDTree across dialogue, code, and mathematical reasoning tasks.

By Shijing Hu, Xuancheng Ren, Zhihui Lu, Pan Zhou

arXiv Computation and Language

Sep 22

TreeSpark: Calibrated, Load-Adaptive Draft Trees for Semi-Autoregressive Speculative Decoding

arXiv:2609.22098v1 Announce Type: new Abstract: Speculative decoding accelerates language-model inference by letting a cheap drafter propose tokens that the target model verifies in parallel. Recent...

By Huapeng Zhou, Huayu Wang, Xinyu Wang

arXiv Machine Learning

Jun 3

TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding

arXiv:2606. 03819v1 Announce Type: new Abstract: One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation.

By Peer Rheinboldt, Fr\'ed\'eric Berdoz, Roger Wattenhofer

efficiency benchmarks

arXiv Machine Learning

Oct 8

Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

The paper introduces the Expected Decoding Rounds (EDR) objective, a theoretical framework that models speculative decoding as a Markov reward process and directly optimizes the number of decoding rounds. EDR weights local rejection costs by state occupancies, eliminating auxiliary hyperparameters and enabling unbiased stochastic optimization via temporal‑difference gradients. Applying EDR to fine‑tune state‑of‑the‑art draft models (DSpark and DFly) improves mean accepted length and outperforms prior training objectives across nine benchmarks in math reasoning, code generation, and chat.

By Yunxiao Zhao, Changxiao Cai

llms fine-tuning benchmarks

Hugging Face Trending Papers

Aug 13

DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees

Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path.

llms diffusion efficiency benchmarks