The paper introduces Decoupled Contrastive Decoding (DCD), a method that separates drafting from verification in contrastive decoding. DCD uses an expert‑aligned lightweight proposer for drafting and applies contrastive correction only during verification, preserving the vanilla contrastive decoding distribution. Experiments on 8B models show that DCD achieves 1.65–1.95× speedups in greedy decoding and reduces MMLU proposal‑path latency by 5–12× compared to amateur‑coupled approaches.
By Zhixuan Liu, Zhichen Dong, Yuanfu Wang, Chao Yang
The paper introduces AdaptiveSpec, a training‑free speculative decoding method that simultaneously adapts the per‑step verification rule and the draft‑tree shape using signals generated during decoding. It replaces the fixed token‑match rule with a margin‑based threshold and adjusts tree depth, width, and node count based on draft confidence and recent acceptance history, allowing the total draft count to vary. Experiments on SGLang show up to 56% throughput gains over EAGLE‑3 while maintaining 93% of lossless task accuracy on GSM8K, MATH‑500, and HumanEval across three models.
By Oszk\'ar Urb\'an, Young D. Kwon, Stylianos I. Venieris, Cecilia Mascolo
arXiv:2610.00623v1 Announce Type: new
Abstract: Speculative decoding has achieved substantial lossless speedups for LLMs, but remains less effective for large vision-language models (LVLMs), where li...
By Wenhan Yang, Anirudh Rao, Ashwin Chandra
arXiv:2606. 26744v1 Announce Type: new Abstract: We present HyperDFlash, a block-parallel speculative decoding framework tailored to the novel multi-hyper-connection (MHC) architecture proposed by DeepSeek-V4.
By Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan, Zhengda Qin, Qiang Wang, Hongjian Sun, Fangmin Chen, Songwei Liu
arXiv:2606. 26744v2 Announce Type: replace Abstract: We present HyperDFlash, a block-parallel speculative decoding framework tailored to DeepSeek-V4's Hyper-Connections (HC).
By Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan, Zhengda Qin, Qiang Wang, Hongjian Sun, Fangmin Chen, Songwei Liu
The paper introduces DPara, a parallel speculative decoding framework that builds on DSpark-style parallel drafters. DPara eliminates the need for probabilistic guesses by precomputing draft representations for every acceptance boundary and using a lightweight autoregressive head to combine verification outcomes with these representations, enabling full parallelization of the backbone forward pass. Experiments on Qwen3-8B and Qwen3-14B across multiple benchmarks demonstrate average speedups of 3.21× and 3.52× over autoregressive decoding, outperforming existing serial and parallel speculative decoding methods.
By Fuliang Liu, Xue Li, Kun Qian, Zhibin Wang, Wanchun Dou, Wenyuan Yu, Chen Tian
arXiv:2607. 24434v1 Announce Type: cross Abstract: Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory.
By Dengke Han
Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory. We target latency-critical single-user settings where routed experts are staged on demand from CPU memory to a GPU or from Flash to a mobile NPU.
arXiv:2610.07822v1 Announce Type: new
Abstract: Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target...
By Shuhao Li, Fanghua Ye, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen
arXiv:2606. 01019v1 Announce Type: cross Abstract: Large Language Model (LLM) generation remains expensive because autoregressive decoding calls the model once for each new token.
By Xin Su, Dawid Majchrowski, Fangyuan Yu, Vanshil Atul Shah, Sebastian Rogawski, Pawel Morkisz, Anahita Bhiwandiwalla, Phillip Howard
arXiv:2611.05724v1 Announce Type: new
Abstract: Speculative decoding accelerates large language model inference with a small drafter, whose per-guess acceptance rate determines the achievable speedup...
By Hoang Anh Duy Le, Huibing Dong, Alireza Khadem, Ziteng Sun, Xiang Song, Yizheng Huang, Soroush Radpour, Anshumali Shrivastava, Ye He
The paper introduces DLoop, a looped speculative decoding technique that adaptively performs multiple drafting stages before verification, allowing a draft model to continue generating tokens while confident. By verifying all accumulated draft tokens together and training the draft model to handle its own hidden states for unverified tokens, DLoop reduces the number of target‑model forward passes needed. Experiments across several speculative decoding methods show wall‑clock speedups of 5–41 % without sacrificing lossless decoding.
By Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han