LoopAhead is a speculative decoding framework designed for looped Transformers, which repeatedly apply the same Transformer layers to improve parameter efficiency. It introduces a lightweight lookahead drafter (LAD) that predicts multiple future tokens using early-loop states, while deeper-loop feedback determines which draft branches to continue or pause, and the full-depth model finalizes token decisions. This approach achieves up to 7.82× faster decoding than traditional autoregressive methods while maintaining the target model’s output distribution.
By Zixuan Liu, Nan Xue, Rui Bao, Yuhang Xu, Xiaofeng Cao, Jiangchao Yao, Zhiyong Chen
arXiv:2609.36590v1 Announce Type: cross
Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...
By Hankun Lin, Patrick Pynadath, Ruqi Zhang
The paper introduces DLoop, a looped speculative decoding technique that adaptively performs multiple drafting stages before verification, allowing a draft model to continue generating tokens while confident. By verifying all accumulated draft tokens together and training the draft model to handle its own hidden states for unverified tokens, DLoop reduces the number of target‑model forward passes needed. Experiments across several speculative decoding methods show wall‑clock speedups of 5–41 % without sacrificing lossless decoding.
By Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, Dongyoon Han
arXiv:2609.17184v1 Announce Type: new
Abstract: Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurren...
By SangLyul Cho, Langqing Cui, Sehoon Kim, Dongsu Han, Insu Han
The paper introduces Packed-SS, a self‑speculative decoding schedule that combines drafting and verification into a single model call, reducing input size to linear in speculative width. By running the diffusion drafter one step ahead, the method keeps an unresolved suffix for the next iteration, avoiding the sequential wait of standard two‑call decoding. Experiments on Nemotron 3B/8B and SetDLM 1.7B demonstrate higher throughput and lower mean wall time under greedy decoding.
By Yuxiao Chen, Changxiao Cai
arXiv:2609.24197v1 Announce Type: new
Abstract: Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification b...
By Weifan Jiang, Krishna Teja Chitty-Venkata, Megan Flynn, Reed Meyerson, Zhenting Qi, Tianyu Wu, Eldar Kurtic, Minlan Yu, Alexandre Marques
ASPIRE introduces a non‑synchronized batched self‑speculative decoding framework for long‑context LLM inference, addressing the memory bottleneck of attention by drafting tokens with sparse attention and verifying them with full attention. It combines a unified mixed forward pass, a lightweight online speculation scheduler that lets each request independently decide when to verify, and an intra‑draft refresh layer that updates the sparse context at every draft step. Experiments on three models and five benchmarks show 1.70–4.58× speedup over autoregressive baselines and a 27% average improvement over the strongest prior self‑speculative methods.
By Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao, Murali Annavaram, Salman Avestimehr
The paper introduces DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores intra-block dependencies in speculative decoding by evaluating multiple candidate tokens for contextual compatibility across adjacent draft positions. DCOCO maintains parallel drafting while enabling contextually informed token selection at low additional cost. Across multiple verifier backbones and benchmarks, DCOCO achieves the highest average throughput, delivering speedups of up to 6.6× over vanilla decoding and outperforming competing methods by 6–23% under concurrent serving loads.
By Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman
The paper introduces AdaptiveSpec, a training‑free speculative decoding method that simultaneously adapts the per‑step verification rule and the draft‑tree shape using signals generated during decoding. It replaces the fixed token‑match rule with a margin‑based threshold and adjusts tree depth, width, and node count based on draft confidence and recent acceptance history, allowing the total draft count to vary. Experiments on SGLang show up to 56% throughput gains over EAGLE‑3 while maintaining 93% of lossless task accuracy on GSM8K, MATH‑500, and HumanEval across three models.
By Oszk\'ar Urb\'an, Young D. Kwon, Stylianos I. Venieris, Cecilia Mascolo
The paper introduces APEX, a learned controller that improves speculative decoding for large language models by dynamically selecting the best speculation strategy and adjusting draft depth during generation. APEX uses request‑level routing among EAGLE‑3, n‑gram, and draft‑model speculation, and block‑level depth adaptation based on causal decoding signals and verifier feedback. Integrated into vLLM and tested with Qwen3‑8B, APEX achieves up to 5.24× speedup over autoregressive decoding while reducing wasted tokens by 41% compared to fixed‑depth n‑gram speculation.
By Manvi Jha, Zach Zhang, Zhichao Xu, Linbo Liu, Sai Muralidhar Jayanthi, Vinayak Arannil
arXiv:2607. 10661v1 Announce Type: cross Abstract: Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks.
By Zipeng Gao, Zhi Zheng, Qingrong Xia, Junda Lin, Ziwei Zhao, Tong Xu, Zhefeng Wang, Enhong Chen
arXiv:2610.20503v1 Announce Type: new
Abstract: Speculative decoding accelerates autoregressive generation by drafting multiple tokens per step, yet feature-based drafters rely on target hidden featu...
By Bohan Sun, Zunhai Su, Xialie Zhuang, Jianjin Li, Ngai Wong, Chuan-Wei Kuo