arXiv Machine Learning June 25, 2026 By WenHung Lee, Jian-Jia Chen, Xiaolin Lin, Pei-Shuo Wang, Chi-Chih Chang, Chun-Che Yang, Ning-Chi Huang, Grace Li Zhang, Kai-Chiang Wu

Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding

Read the original on arXiv Machine Learning →

arXiv:2606. 24957v1 Announce Type: cross Abstract: While speculative decoding improves inference throughput for multi-batch long-context Large Language Models (LLMs), its efficiency is often limited by a verification bottleneck where Key-Value (KV) cache loading dominates latency.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Sep 1

SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification

arXiv:2512.02337v2 Announce Type: replace Abstract: Growing demands from tasks like code generation, deep reasoning, and long-document understanding have made long-context generation a crucial capabi...

By Zhendong Tan, Xingjun Zhang, Chaoyi Hu, Junjie Peng, Kun Xia

llms benchmarks

arXiv Machine Learning

4d ago

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...

By Hankun Lin, Patrick Pynadath, Ruqi Zhang

llms fine-tuning benchmarks

arXiv Machine Learning

Aug 24

STS: Efficient Sparse Attention with Speculative Token Sparsity

arXiv:2605.15508v3 Announce Type: replace Abstract: The quadratic complexity of attention imposes severe memory and computational bottlenecks on Large Language Model (LLM) inference. This challenge i...

By Jiangnan Yu, Ceyu Xu, Yongji Wu, Yuan Xie

llms agents efficiency benchmarks

arXiv Machine Learning

Jun 2

Vegas: Self-Speculative Decoding with Verification-Guided Sparse Attention

arXiv:2602. 07223v2 Announce Type: replace Abstract: Long-context large language model (LLM) inference has become the norm for today's AI applications.

By Yikang Yue, Yuqi Xue, Jian Huang

llms efficiency benchmarks

arXiv AI

4d ago

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

arXiv:2609.37029v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context...

By Hao-Yuan He, Peng-Fei Liu, Si Shen, Ming Li

llms diffusion benchmarks

arXiv AI

Jun 3

KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem

arXiv:2602. 20217v2 Announce Type: replace-cross Abstract: Self-speculative decoding (SSD) accelerates LLM inference by skipping layers to create an efficient draft model, yet existing methods often rely on static heuristics that ignore the dynamic computational overhead of attention in long-context scenarios.

By Seongjin Cha, Gyuwan Kim, Dongsu Han, Tao Yang, Insu Han

llms efficiency benchmarks

One email a morning, machine-written

Related stories

SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

STS: Efficient Sparse Attention with Speculative Token Sparsity

Vegas: Self-Speculative Decoding with Verification-Guided Sparse Attention

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem