arXiv AI October 12, 2026 By Raghavv Goel, Sudhanshu Agrawal, Mukul Gagrani, Risheek Garrepalli, Mingu Lee

Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding

The paper introduces a principled framework for selecting multiple hidden-state layers from large language model decoders to improve speculative decoding. It uses matrix‑entropy to gauge informativeness and centered kernel alignment (CKA) to assess diversity, revealing distinct geometric and informational compartments across layers. The authors propose entropy‑ and CKA‑based selection algorithms, token‑entropy filtering, and demonstrate that these choices outperform traditional fixed‑layer heuristics in EAGLE3‑style decoding.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 30

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...

By Hankun Lin, Patrick Pynadath, Ruqi Zhang

llms fine-tuning benchmarks

arXiv AI

Oct 27

T2D: Target-to-Draft Model Pruning & Adaptation for Efficient Speculative Decoding

The paper introduces T2D, a framework that prunes and adapts a target large language model into a draft model optimized for both inference latency and alignment with the target model’s outputs. By formulating draft model design as an optimization problem, T2D achieves up to 92% pruning while maintaining high speculative decoding performance across multiple target model families and domains. The resulting draft models can also be integrated with other speculative decoding techniques to yield additional gains.

By Shahriar Kabir Nahin, Xiao Ye, Muhao Chen, Ben Zhou, Anshuman Chhabra

llms efficiency benchmarks safety

arXiv AI

Sep 1

Entropy-Aware Token Rejection for Improving Speculative Decoding

arXiv:2512.23765v2 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronge...

By Tiancheng Su, Meicong Zhang, Guoxiu He

llms benchmarks

arXiv AI

Jun 2

SimSD: Simple Speculative Decoding in Diffusion Language Models

arXiv:2606. 02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding.

By Junxia Cui, Haotian Ye, Runchu Tian, Hongcan Guo, Jinya Jiang, Haoru Li, Chaojie Ren, Yiming Huang, Kaijie Zhu, Zhongkai Yu, Kun Zhou, Jingbo Shang

llms diffusion efficiency benchmarks

arXiv Computation and Language

Sep 1

ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding

arXiv:2608.29748v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...

By Luxi Lin, Zhanpeng Zeng, Shuang Peng, Songwei Liu, Rongrong Ji

llms rag diffusion

arXiv Machine Learning

Jun 30

Depth Exploration for LLM Decoding

arXiv:2606. 29223v1 Announce Type: new Abstract: Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths.

By Weisi Yang, Zipeng Sun, Stephen Xia

llms efficiency

One email a morning, machine-written

Related stories

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

T2D: Target-to-Draft Model Pruning & Adaptation for Efficient Speculative Decoding

Entropy-Aware Token Rejection for Improving Speculative Decoding

SimSD: Simple Speculative Decoding in Diffusion Language Models

ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding

Depth Exploration for LLM Decoding