arXiv Machine Learning
ProVe: Proposer-Verifier Decoding
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Oct 15
JetSpec++: Foresight Parallel Drafting with Self Refinements
arXiv:2610.19792v1 Announce Type: new Abstract: Speculative decoding's effectiveness is governed by two quantities: the acceptance rate of drafted tokens and the cost of producing them. We show the t...
arXiv Machine Learning
Oct 7
APEX: Speculate smarter, not deeper
The paper introduces APEX, a learned controller that improves speculative decoding for large language models by dynamically selecting the best speculation strategy and adjusting draft depth during generation. APEX uses request‑level routing among EAGLE‑3, n‑gram, and draft‑model speculation, and block‑level depth adaptation based on causal decoding signals and verifier feedback. Integrated into vLLM and tested with Qwen3‑8B, APEX achieves up to 5.24× speedup over autoregressive decoding while reducing wasted tokens by 41% compared to fixed‑depth n‑gram speculation.
arXiv AI
Sep 15
Carryover Drafting: Recycling Rejected States for Speculative Decoding
arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept seve...
arXiv AI
Jun 11
VIA-SD: Verification via Intra-Model Routing for Speculative Decoding
arXiv:2606. 12243v1 Announce Type: cross Abstract: Speculative decoding (SD) addresses the high inference costs of LLMs by having lightweight drafters generate candidates for large verifiers to validate in parallel.
arXiv AI
Aug 5
Approximate Speculative Decoding
arXiv:2608. 03447v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel.
arXiv AI
Aug 26
ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping
ResiSpec is a framework that improves speculative decoding for large language models by reshaping the residual distribution during verification. It addresses the problem of residual drift, where rejected candidates cause the target distribution to diverge from the draft model’s predictions, rendering later candidates ineffective. By aligning the verification process with the draft model’s high‑confidence regions, ResiSpec prevents candidate obsolescence and achieves up to 1.92× speedup over existing multi‑candidate methods.