arXiv AI November 5, 2026 By Geng Zhang, Fangzhao Zhang, Cong Xie, Tiantian Fan, Yang You

Split-KV: Efficient Context Parallelism for Training Diffusion Drafters

Split-KV introduces a context‑parallelism technique for training diffusion drafters, keeping key/value shards local and limiting per‑layer attention communication to query‑sized tensors. This approach exploits the asymmetry between short draft queries and long contexts, enabling efficient load balancing via a cost‑aware longest‑processing‑time‑first strategy. Experiments show that at a CP size of 64, Split‑KV achieves 85–91 % weak‑scaling efficiency and up to 3.67× speedup over the best baseline, with less than 1 % load imbalance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

diffusion

arXiv AI

Sep 30

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

arXiv:2609.37029v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context...

By Hao-Yuan He, Peng-Fei Liu, Si Shen, Ming Li

llms diffusion benchmarks

arXiv Computation and Language

Oct 12

Contextual Correction of Parallel Draftsfor Speculative Decoding

The paper introduces DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores intra-block dependencies in speculative decoding by evaluating multiple candidate tokens for contextual compatibility across adjacent draft positions. DCOCO maintains parallel drafting while enabling contextually informed token selection at low additional cost. Across multiple verifier backbones and benchmarks, DCOCO achieves the highest average throughput, delivering speedups of up to 6.6× over vanilla decoding and outperforming competing methods by 6–23% under concurrent serving loads.

By Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman

llms diffusion efficiency benchmarks

arXiv Machine Learning

Sep 21

Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding

Elastic Threshold Attention (ETA) is a trainable attention mechanism that dynamically predicts contextual thresholds from query representations, enabling selective pruning of KV cache tokens during long‑context decoding. By multiplicatively suppressing sub‑threshold logits during training, ETA avoids representation collapse and eliminates localized attention sinks, allowing a 1.45B model to match dense attention performance at roughly 85% training sparsity and 38% active decode density. At inference, a custom Triton kernel achieves up to 2.5× faster decoding on sequences up to 512K tokens, and an offline calibration step can further reduce compute by 27% by freezing per‑head thresholds.

By Themistoklis Haris, Henry Li, Maryam Karimzadehgan

efficiency

arXiv Machine Learning

Sep 22

H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache

arXiv:2609.24197v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification b...

By Weifan Jiang, Krishna Teja Chitty-Venkata, Megan Flynn, Reed Meyerson, Zhenting Qi, Tianyu Wu, Eldar Kurtic, Minlan Yu, Alexandre Marques

llms diffusion efficiency

arXiv AI

Jun 2

BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding

arXiv:2606. 00144v1 Announce Type: cross Abstract: Speculative decoding speeds up autoregressive decoding by using a drafter to propose multiple tokens that a verifier validates in parallel.

By Liang He, Jingbo Wen, Qishi Zhan, Yixiong Chen, Kangning Cui, Qizhen Lan, Xilu Wang

efficiency

Hugging Face Trending Papers

Oct 13

Prefilling-dLLM v2: Learned Chunk Routing for Long-Context Diffusion Language Models

Prefilling-dLLM v2 is a diffusion language model that learns to route computation using a native mask token, enabling efficient sparse attention across long contexts. By appending a mask at the end of each chunk and using a learned role offset, the model selectively attends to top‑K chunks every fourth layer while maintaining local windows elsewhere. Evaluated on 16K–128K contexts, Prefilling-dLLM v2 outperforms other sparse methods (NSA, DSA, SWA) on RULER tasks and achieves a 172× speedup over full dense attention at 128K tokens.

llms diffusion