The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Orthrus is a hybrid autoregressive‑diffusion model that claims to perform lossless speculative decoding by generating multiple tokens in parallel while maintaining a frozen autoregressive backbone. In our independent reproduction, we found that under BF16 precision the model matches the exact autoregressive trajectory only about 45% of the time, whereas with FP32 precision it achieves perfect matching on all evaluated prompts. Despite trajectory mismatches at lower precision, Orthrus does not exhibit systematic degradation on downstream lm‑eval‑harness benchmarks.
arXiv:2606. 27474v1 Announce Type: cross Abstract: How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding?
arXiv:2608. 08721v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round.
ResiSpec is a framework that improves speculative decoding for large language models by reshaping the residual distribution during verification. It addresses the problem of residual drift, where rejected candidates cause the target distribution to diverge from the draft model’s predictions, rendering later candidates ineffective. By aligning the verification process with the draft model’s high‑confidence regions, ResiSpec prevents candidate obsolescence and achieves up to 1.92× speedup over existing multi‑candidate methods.
The paper introduces TSS, a target-side sparsification framework that selectively skips layers in a target verifier during speculative decoding for domain-specific large language models. By exploring multi-layer skip configurations with an acceptance- and metric-aware breadth search, TSS reduces verification cost, increases draft acceptance, and can even improve downstream task performance without retraining. Experiments on Spec-Bench demonstrate consistent gains across domains and model scales, notably boosting translation throughput by 1.68× and improving BLEU scores significantly.
The paper introduces a trajectory-level speculative decoding framework for diffusion-based language models (dLLMs), addressing the limitation of existing strategies that revert to single-token generation when confidence is low. By constructing draft denoising trajectories through confidence-stratified tree exploration and verifying them with blockwise parallel evaluation and bidirectional attention masking, the method also incorporates inter-block speculation to exploit the models’ bidirectional structure. Experiments show a 30–40% reduction in denoising iterations, a token-per-step increase from 2.6 to 4.3, and a 7–14× speedup over vanilla dLLMs while maintaining accuracy within 1% on reasoning and code benchmarks.