arXiv Machine Learning October 28, 2026 By Yuxuan Xia, Wenbo Lu, Shaoyi Zheng, Shengjie Wang

INTRA: Interleaved Non-contiguous Token spaRse Attention

Read the original on arXiv Machine Learning →

The paper introduces INTRA, a hardware‑aware static sparse attention mechanism that interleaves non‑contiguous token groups to enable efficient long‑range communication while reducing quadratic cost. INTRA uses Scatter and Gather layers to create short mediated paths across layers, and a FlashAttention‑style kernel computes structured indices at runtime for dense packing of selected tokens. Experiments on an A100 GPU show significant speedups—up to 8.4× over MoBA and 10.4× over density‑constrained NSA—while maintaining or improving image quality and long‑context performance on models like FLUX.1‑dev and LLaMA‑3.1‑8B.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Oct 13

Joint Subspace Sparse Attention for Efficient Long-Context Decoding

Sparse attention reduces the KV-cache traffic that dominates long-context LLM decoding, but practical methods largely operate at coarse granularity. Token-level selection offers finer control, yet pre...

llms efficiency

arXiv Computation and Language

Oct 15

Joint Subspace Sparse Attention for Efficient Long-Context Decoding

Joint Subspace Sparse Attention (JSSA) is a new method that reduces KV‑cache traffic in long‑context LLM decoding by selecting tokens at the token level while preserving query–key structure. It learns a shared low‑dimensional query–key subspace through a tractable spectral solution and implements a gather‑free indexed‑attention kernel that supports per‑KV‑head selections. Across various attention architectures, JSSA maintains near full‑KV accuracy, avoids generation‑length inflation, and can achieve up to 6.43× decoding throughput on a single GPU for 128K context, even at 700B‑parameter scale.

By Jiwon Song, Yoongon Kim, Jae-Joon Kim

llms efficiency

arXiv AI

Jun 12

MiniMax Sparse Attention

arXiv:2606. 13392v1 Announce Type: new Abstract: Ultra-long-context capability is becoming indispensable for frontier LLMs: agentic workflows, repository-scale code reasoning, and persistent memory all require the model to jointly attend over hundreds of thousands to millions of tokens, yet the quadratic cost of softmax attention makes this untenable at deployment scale.

By Xunhao Lai, Weiqi Xu, Yufeng Yang, Qiaorui Chen, Yang Xu, Lunbin Zeng, Xiaolong Li, Haohai Sun, Haichao Zhu, Vito Zhang, Pengyu Zhao

llms agents efficiency multimodal

arXiv Machine Learning

Jun 4

SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference

arXiv:2606. 04511v1 Announce Type: cross Abstract: Sparse attention reduces compute and memory bandwidth for long-context LLM inference.

By Yaosheng Fu, Guangxuan Xiao, Xin Dong, Song Han, Oreste Villa

llms efficiency

arXiv Machine Learning

Jul 2

FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel

arXiv:2508. 18224v3 Announce Type: replace-cross Abstract: Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language models (LLMs).

By Ran Yan, Youhe Jiang, Zhuoming Chen, Haohui Mai, Beidi Chen, Binhang Yuan

llms benchmarks

arXiv Machine Learning

Jun 5

You Only Index Once: Cross-Layer Sparse Attention with Shared Routing

arXiv:2606. 06467v1 Announce Type: cross Abstract: Long-context inference in modern LLMs is increasingly constrained by decoding efficiency, especially in reasoning-heavy settings where models generate long intermediate chains of thought.

By Yutao Sun, Yanqi Zhang, Li Dong, Jianyong Wang, Furu Wei

llms efficiency benchmarks

One email a morning, machine-written

Related stories

Joint Subspace Sparse Attention for Efficient Long-Context Decoding

Joint Subspace Sparse Attention for Efficient Long-Context Decoding

MiniMax Sparse Attention

SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference

FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel

You Only Index Once: Cross-Layer Sparse Attention with Shared Routing