Orthrus is a hybrid autoregressive‑diffusion model that claims to perform lossless speculative decoding by generating multiple tokens in parallel while maintaining a frozen autoregressive backbone. In our independent reproduction, we found that under BF16 precision the model matches the exact autoregressive trajectory only about 45% of the time, whereas with FP32 precision it achieves perfect matching on all evaluated prompts. Despite trajectory mismatches at lower precision, Orthrus does not exhibit systematic degradation on downstream lm‑eval‑harness benchmarks.
arXiv:2606. 27474v1 Announce Type: cross Abstract: How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding?
ResiSpec is a framework that improves speculative decoding for large language models by reshaping the residual distribution during verification. It addresses the problem of residual drift, where rejected candidates cause the target distribution to diverge from the draft model’s predictions, rendering later candidates ineffective. By aligning the verification process with the draft model’s high‑confidence regions, ResiSpec prevents candidate obsolescence and achieves up to 1.92× speedup over existing multi‑candidate methods.
arXiv:2608. 08721v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round.
The paper introduces a trajectory-level speculative decoding framework for diffusion-based language models (dLLMs), addressing the limitation of existing strategies that revert to single-token generation when confidence is low. By constructing draft denoising trajectories through confidence-stratified tree exploration and verifying them with blockwise parallel evaluation and bidirectional attention masking, the method also incorporates inter-block speculation to exploit the models’ bidirectional structure. Experiments show a 30–40% reduction in denoising iterations, a token-per-step increase from 2.6 to 4.3, and a 7–14× speedup over vanilla dLLMs while maintaining accuracy within 1% on reasoning and code benchmarks.
arXiv:2606. 02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding.
arXiv:2607. 05147v1 Announce Type: new Abstract: Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification.
arXiv:2607. 19223v1 Announce Type: new Abstract: Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference.
arXiv:2608.29748v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...
arXiv:2607. 10661v1 Announce Type: cross Abstract: Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks.
The paper examines lossy verification techniques used in speculative decoding for large language models, showing that many methods can be grouped into truncation-based and collaborative verification categories. It analyzes how these approaches alter the decoding distribution, revealing that truncation-based methods can significantly degrade performance due to distributional distortion, while collaborative methods depend more on overshoot suppression and supervision quality than on simple interpolation between draft and target models. A diagnostic evaluation framework is introduced to assess these failure modes across curated benchmarks.
The paper introduces AdaptiveSpec, a training‑free speculative decoding method that simultaneously adapts the per‑step verification rule and the draft‑tree shape using signals generated during decoding. It replaces the fixed token‑match rule with a margin‑based threshold and adjusts tree depth, width, and node count based on draft confidence and recent acceptance history, allowing the total draft count to vary. Experiments on SGLang show up to 56% throughput gains over EAGLE‑3 while maintaining 93% of lossless task accuracy on GSM8K, MATH‑500, and HumanEval across three models.
Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference. Recent work such as DFlash further boosts drafting efficiency by leveraging diffusion drafters, whose parallel denoising mechanism enables draft generation in a single forward pass.