The Flow has not summarised this story yet — read it at arXiv Computation and Language.
While draft-model-free speculative decoding offers a promising path to efficient LLM inference, it is frequently constrained by stale draft candidates and the high computational cost of the verificati...
LoopAhead is a speculative decoding framework designed for looped Transformers, which repeatedly apply the same Transformer layers to improve parameter efficiency. It introduces a lightweight lookahead drafter (LAD) that predicts multiple future tokens using early-loop states, while deeper-loop feedback determines which draft branches to continue or pause, and the full-depth model finalizes token decisions. This approach achieves up to 7.82× faster decoding than traditional autoregressive methods while maintaining the target model’s output distribution.
arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...
WaveFront Decoding (WFD) is a training‑free, self‑speculative decoding framework for looped language models that reduces decoding latency by batching draft and verification steps within the same recurrent‑block calls. By exploiting intermediate recurrence outputs as draft predictions and weight sharing to process token states at different depths together, WFD arranges mixed‑depth states into a diagonal wavefront, allowing shallow‑depth drafting while deeper‑depth verification proceeds concurrently. Experiments on six Spec‑Bench task categories show WFD achieving up to 4.81× speedup on Huginn‑3.5B compared to autoregressive decoding, outperforming traditional draft‑then‑verify approaches.
arXiv:2609.24698v1 Announce Type: new Abstract: Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which foll...
arXiv:2609.17184v1 Announce Type: new Abstract: Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurren...