arXiv Machine Learning
LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 16
LoopAhead: Speculating Ahead for Faster Decoding in Looped Language Models
LoopAhead is a speculative decoding framework designed for looped Transformers, which repeatedly apply the same Transformer layers to improve parameter efficiency. It introduces a lightweight lookahead drafter (LAD) that predicts multiple future tokens using early-loop states, while deeper-loop feedback determines which draft branches to continue or pause, and the full-depth model finalizes token decisions. This approach achieves up to 7.82× faster decoding than traditional autoregressive methods while maintaining the target model’s output distribution.
arXiv Machine Learning
Sep 30
SEED: Self-Speculative Decoding via Implicit Encoder-Decoder
arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...
arXiv Machine Learning
Sep 22
WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models
WaveFront Decoding (WFD) is a training‑free, self‑speculative decoding framework for looped language models that reduces decoding latency by batching draft and verification steps within the same recurrent‑block calls. By exploiting intermediate recurrence outputs as draft predictions and weight sharing to process token states at different depths together, WFD arranges mixed‑depth states into a diagonal wavefront, allowing shallow‑depth drafting while deeper‑depth verification proceeds concurrently. Experiments on six Spec‑Bench task categories show WFD achieving up to 4.81× speedup on Huginn‑3.5B compared to autoregressive decoding, outperforming traditional draft‑then‑verify approaches.
arXiv Computation and Language
Oct 27
LoopFold: Parallelizing Recurrent Depth in Looped Transformers
arXiv:2610.35556v1 Announce Type: new Abstract: Looped Transformers increase computation per token by repeatedly applying a shared backbone, offering a way to improve language modeling and downstream...
arXiv AI
Oct 31
CopyLoop: Fast Looped Transformer Decoding via Context Copying and Shallow Screening
CopyLoop is a training‑free decoding algorithm for looped transformers that speeds up generation by proposing multi‑token suffixes from repeated context, evaluating them early at a shallow recurrent depth, and scheduling continuous recurrent processing. It avoids step‑by‑step decoding, allowing newer blocks to be processed concurrently while earlier blocks continue toward full‑depth verification. Across six model checkpoints from the Ouro, Raven, and Huginn families, CopyLoop delivers 2.98x–10.70x speedups on Base models and 2.74x–3.52x on Thinking models, outperforming or matching all competing baselines on code generation and multi‑step mathematical reasoning tasks.
arXiv Machine Learning
Jul 13
Accelerating Large Language Model Inference with Self-Supervised Early Exits
arXiv:2407. 21082v3 Announce Type: replace-cross Abstract: This paper presents a modular approach to accelerate inference in large language models (LLMs) by adding early exit heads at intermediate transformer layers.