arXiv Machine Learning July 13, 2026 By Florian Valade

Accelerating Large Language Model Inference with Self-Supervised Early Exits

Read the original on arXiv Machine Learning →

arXiv:2407. 21082v3 Announce Type: replace-cross Abstract: This paper presents a modular approach to accelerate inference in large language models (LLMs) by adding early exit heads at intermediate transformer layers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

4d ago

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...

By Hankun Lin, Patrick Pynadath, Ruqi Zhang

llms fine-tuning benchmarks

arXiv AI

Jun 4

SSSD: Simply-Scalable Speculative Decoding

arXiv:2411. 05894v3 Announce Type: replace-cross Abstract: Speculative Decoding has emerged as a popular technique for accelerating inference in Large Language Models.

By Michele Marzollo, Jiawei Zhuang, Niklas Roemer, Niklas Zwingenberger, Lorenz K. M\"uller, Lukas Cavigelli

llms benchmarks

arXiv Machine Learning

Sep 16

LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

arXiv:2609.17184v1 Announce Type: new Abstract: Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurren...

By SangLyul Cho, Langqing Cui, Sehoon Kim, Dongsu Han, Insu Han

llms benchmarks

arXiv AI

Jul 14

Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting

arXiv:2607. 10661v1 Announce Type: cross Abstract: Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks.

By Zipeng Gao, Zhi Zheng, Qingrong Xia, Junda Lin, Ziwei Zhao, Tong Xu, Zhefeng Wang, Enhong Chen

llms efficiency benchmarks

Hugging Face Trending Papers

Jul 12

Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting

Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks. However, traditional speculative decoding typically relies on auxiliary draft modules, incurring significant training and communication overhead.

llms efficiency benchmarks

arXiv AI

Sep 1

Entropy-Aware Token Rejection for Improving Speculative Decoding

arXiv:2512.23765v2 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronge...

By Tiancheng Su, Meicong Zhang, Guoxiu He

llms benchmarks

One email a morning, machine-written

Related stories

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

SSSD: Simply-Scalable Speculative Decoding

LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting

Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting

Entropy-Aware Token Rejection for Improving Speculative Decoding