arXiv AI July 21, 2026 By Junhao Hu, Fangze Li, Mingtao Xu, Feifan Meng, Shiju Zhao, Tiancheng Hu, Ting Peng, Anmin Liu, Wenrui Huang, Chenxu Liu, Ziyue Hua, Tao Xie

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage

Read the original on arXiv AI →

arXiv:2601. 03043v4 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate strong capabilities across a wide range of complex tasks and are increasingly deployed at scale, placing significant demands on inference efficiency.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 9

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

arXiv:2605. 16928v2 Announce Type: replace-cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention.

By Yanke Zhou, Yiduo Li, Hanlin Tang, Maohua Li, Kan Liu, Tao Lan, Lin Qu, Yuan Yao, Xiaoxing Ma

llms efficiency benchmarks

arXiv Computation and Language

Sep 1

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

arXiv:2606.10537v2 Announce Type: replace Abstract: Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with...

By Jing Xiong, Qi Han, Shansan Gong, Yunta Hsieh, Boyuan Zheng, Chengyue Wu, Chaofan Tao, Chenyang Zhao, Ngai Wong

llms diffusion efficiency benchmarks

arXiv Machine Learning

Aug 24

STS: Efficient Sparse Attention with Speculative Token Sparsity

arXiv:2605.15508v3 Announce Type: replace Abstract: The quadratic complexity of attention imposes severe memory and computational bottlenecks on Large Language Model (LLM) inference. This challenge i...

By Jiangnan Yu, Ceyu Xu, Yongji Wu, Yuan Xie

llms agents efficiency benchmarks

arXiv Machine Learning

Jun 24

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs

arXiv:2504. 17768v3 Announce Type: replace-cross Abstract: Sparse attention offers a promising strategy to extend long-context capabilities in Transformer LLMs, yet its efficiency-accuracy trade-offs remain unclear due to the lack of comprehensive evaluation.

By Piotr Nawrot, Robert Li, Renjie Huang, Sebastian Ruder, Kelly Marchisio, Edoardo M. Ponti

llms efficiency

Hugging Face Trending Papers

Jun 9

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long-context scenarios. We propose Prefilling-dLLM, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per-step complexity from quadratic in the full sequence length to quadratic only in the decode length.

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jun 29

Simplified Sparse Attention via Gist Tokens

arXiv:2604. 20920v2 Announce Type: replace Abstract: Sparse attention can reduce the cost of long-context inference, but most variants introduce new architectural components.

By Yuzhen Mao, Michael Y. Li, Emily B. Fox

llms rag efficiency

One email a morning, machine-written

Related stories

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

STS: Efficient Sparse Attention with Speculative Token Sparsity

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

Simplified Sparse Attention via Gist Tokens