Hugging Face Trending Papers August 31, 2026

Strong Drafts Need Compact Memories: Long-Context Speculative Decoding with Compressed KV Cache

Read the original on Hugging Face Trending Papers →

The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.

arXiv Machine Learning

Sep 1

Strong Drafts Need Compact Memories: Long-Context Speculative Decoding with Compressed KV Cache

arXiv:2608.30252v1 Announce Type: new Abstract: Long-context LLM applications such as document summarization and multi-turn agents require generation from prefixes spanning tens of thousands of token...

By Tong Yuan, Chengxi Liao, Zeyi Wen

llms agents nlp efficiency

arXiv Machine Learning

Sep 22

H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache

arXiv:2609.24197v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification b...

By Weifan Jiang, Krishna Teja Chitty-Venkata, Megan Flynn, Reed Meyerson, Zhenting Qi, Tianyu Wu, Eldar Kurtic, Minlan Yu, Alexandre Marques

llms diffusion efficiency

arXiv Machine Learning

Oct 15

FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding

arXiv:2610.20392v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM generation through lightweight drafting and parallel verification. It is particularly promising for long inputs,...

By Shuxuan Li, Jialiang Kang, Yanghao Zhou, Ziye Zhu, Xiao Yang

llms benchmarks

arXiv AI

Sep 30

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

arXiv:2609.37029v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context...

By Hao-Yuan He, Peng-Fei Liu, Si Shen, Ming Li

llms diffusion benchmarks

arXiv AI

Oct 5

BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration

arXiv:2610.02800v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by using a lightweight draft to propose multiple tokens for parallel verification. However,...

By Chence Yang, Ningxi Cheng, Arash Akbari, Qitao Tan, Qingchan Zhu, Ci Zhang, Changdi Yang, Yanzhi Wang, Wei Niu, Jinhui Wang, Jin Lu, Geng Yuan

llms efficiency

arXiv Machine Learning

Sep 30

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...

By Hankun Lin, Patrick Pynadath, Ruqi Zhang

llms fine-tuning benchmarks

One email a morning, machine-written

Related stories

Strong Drafts Need Compact Memories: Long-Context Speculative Decoding with Compressed KV Cache

H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache

FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder