The paper introduces CADENCE, a context‑adaptive decoding method that learns both what to draft and how far to draft during language generation. It uses Gaussian prediction heads to encode future token spans as continuous vectors, a shared decoder to produce tokens in parallel, and a learned router to choose the prediction length based on hidden states and Gaussian parameters. Experiments on Llama‑3.1‑8B‑Instruct show that CADENCE’s router‑selected positions achieve higher draft acceptance and up to 1.3× higher generation throughput compared to standard greedy decoding.
arXiv:2610.28619v1 Announce Type: new Abstract: Multi-token prediction (MTP) can accelerate language-model decoding by drafting several tokens and verifying them in a single target-model pass. Howeve...
The paper introduces DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores intra-block dependencies in speculative decoding by evaluating multiple candidate tokens for contextual compatibility across adjacent draft positions. DCOCO maintains parallel drafting while enabling contextually informed token selection at low additional cost. Across multiple verifier backbones and benchmarks, DCOCO achieves the highest average throughput, delivering speedups of up to 6.6× over vanilla decoding and outperforming competing methods by 6–23% under concurrent serving loads.
arXiv:2609.38510v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusi...
The paper introduces LS-AR, a dual‑channel architecture that separates continuous goal steering from discrete token decoding using FiLM conditioning. It employs a Static Goal Encoder for long‑term macro‑objective retention and a Dynamic State Tracker for recurrent latent updates. Experiments show LS‑AR achieves perfect target recall on long‑horizon retrieval tasks, improves throughput by ~35%, reduces VRAM usage by 52.8%, and maintains higher completion rates in Blocksworld planning under perturbations.
arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...
arXiv:2606. 27550v1 Announce Type: cross Abstract: Multi-token prediction has been shown to increase data density during training, improve downstream text-generation quality, and serves as the defacto approach for self-speculative decoding.
TeCoP is a training‑free decoder that accelerates diffusion large language model (dLLM) decoding by decoupling prediction readiness from context utility. It uses tentative context pipelining and draft‑and‑verify to reuse stable but uncommitted predictions as provisional cross‑block context, with commitment based on confidence and stability checks. Across four dLLMs on GSM8K, Minerva MATH, and MBPP, TeCoP cuts forward evaluations by up to 87% and boosts throughput up to 7.5‑fold without KV caching and 11.77‑fold with caching, outperforming Fast‑dLLM and DSB while maintaining accuracy within one percentage point.
arXiv:2609.37029v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context...
arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.
The paper introduces APEX, a learned controller that improves speculative decoding for large language models by dynamically selecting the best speculation strategy and adjusting draft depth during generation. APEX uses request‑level routing among EAGLE‑3, n‑gram, and draft‑model speculation, and block‑level depth adaptation based on causal decoding signals and verifier feedback. Integrated into vLLM and tested with Qwen3‑8B, APEX achieves up to 5.24× speedup over autoregressive decoding while reducing wasted tokens by 41% compared to fixed‑depth n‑gram speculation.
arXiv:2609.01068v1 Announce Type: new Abstract: The heavy-tailed distribution of output lengths in Large Language Model (LLM) serving poses major challenges for resource provisioning and cluster sche...
arXiv:2610.20503v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by drafting multiple tokens per step, yet feature-based drafters rely on target hidden featu...