arXiv AI
Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
6d ago
DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification
arXiv:2609.37532v1 Announce Type: cross Abstract: Growing large language model applications demand efficient inference. At high concurrency, block-diffusion speculative decoding suffers from verifica...
arXiv AI
Jun 29
Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks
arXiv:2606. 27474v1 Announce Type: cross Abstract: How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding?
Towards Data Science
Aug 24
Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash
Speculative decoding leverages idle CPU resources to accelerate token generation without altering model outputs. In vLLM benchmarks, DFlash achieved a 3.92× increase in autoregressive throughput using Qwen3.5‑9B on an Intel Xeon 6 at a concurrency of 1. The article details the origins of this speedup, discusses acceptance metrics, and outlines factors that influence when speculation is beneficial.
arXiv AI
6d ago
Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding
arXiv:2609.36173v1 Announce Type: cross Abstract: Parallel speculative drafting generates multiple candidates in one backbone pass, but independent token selection can produce inconsistent continuati...
arXiv AI
Jun 2
MineDraft: A Framework for Batch Parallel Speculative Decoding
arXiv:2603. 18016v2 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates large language model inference by using a smaller draft model to propose draft tokens that are subsequently verified by a larger target model.
arXiv AI
Jul 7
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
arXiv:2607. 05147v1 Announce Type: new Abstract: Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification.