arXiv:2608.30427v1 Announce Type: cross Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, pres...
arXiv:2608. 13524v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel.
arXiv:2608.20375v1 Announce Type: new Abstract: Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree b...
TreeWY introduces a speculative verification method for Gated DeltaNet (GDN) hybrid models that eliminates the need for per-draft-state snapshots. By applying a tree‑structured WY transform to the gated delta rule, each draft node’s output is computed with a single triangular solve, and only the accepted state is reconstructed on commit. Benchmarks on Qwen3.5 35B and 397B show reduced memory pressure, higher throughput, and lower time‑to‑first‑token in memory‑bound scenarios, while enabling wider, higher‑acceptance draft trees.
arXiv:2609.23900v1 Announce Type: new Abstract: Tree speculative decoding verifies multiple candidate continuations in one target forward pass. For attention-only transformers, the verifier mainly ne...
Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path.
The paper introduces Tree Exit Verification (TEV), an exact, level‑parallel speculative decoding method that uses a single exit‑node decision and a bonus‑token decision to verify draft continuations. By deriving a canonical exit law, the authors show that TEV achieves the same acceptance as optimal verifiers while reducing verifier‑stage latency by 15%. They also propose Exit‑Guided Draft‑Tree Training (ExitTrain), which improves draft trees and increases average output‑block length by 13%, leading to a 14% end‑to‑end speedup over DDTree across dialogue, code, and mathematical reasoning tasks.
arXiv:2609.22098v1 Announce Type: new Abstract: Speculative decoding accelerates language-model inference by letting a cheap drafter propose tokens that the target model verifies in parallel. Recent...
arXiv:2608.20530v1 Announce Type: new Abstract: Speculative decoding accelerates language-model inference by drafting future tokens that the target model verifies in parallel. A diffusion-style block...
arXiv:2609.36173v1 Announce Type: cross Abstract: Parallel speculative drafting generates multiple candidates in one backbone pass, but independent token selection can produce inconsistent continuati...
PreSTO introduces Predictive SubTree Prefetching to accelerate PowerMH, a power sampling method for large language models. By prefetching future proposals in a tree of accept/reject decisions and using a predictive traversal rule, PreSTO maintains the same Metropolis–Hastings transition kernel while reducing model calls. Experiments on seven datasets and five base LLMs show median speedups of 1.71×, 1.91×, and 2.33× for PowerMH, EntropyCut, and MultiTryMH, respectively, with fewer and shorter model calls.
On-Policy Residual Learning for Diffusion Language Models (ORL) introduces a framework that learns residual context directly from model-generated trajectories, using an entropy-contraction objective to sharpen predictions. The approach co-designs algorithmic and system components—such as asynchronous residual capture, block-diffusion attention, and packed compute-balanced training—to enable practical scaling. Experiments on a 4B DLM converted from Qwen3-4B show up to 6.2× faster tokens per forward and 6.9× higher end-to-end throughput, while also improving speed-quality trade-offs over autoregressive models and transferring effectively to other DLMs like LLaDA-2.1-mini and DiffusionGemma.