← Back to all news
arXiv Machine Learning By Yufei Xu, Fanxu Meng, Fan Jiang, Yuxuan Wang, Ruijie Zhou, Zhaohui Wang, Jiexi Wu, Zhixin Pan, Xiaojuan Tang, Wenjie Pei, Tongxuan Liu, Di Yin, Xing Sun, Muhan Zhang

HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention

Read the original on arXiv Machine Learning →

HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention proposes a two-stage hierarchical indexer that replaces the flat token scan used in token-level sparse attention mechanisms like DeepSeek Sparse Attention. The method first performs block-level coarse filtering to discard irrelevant regions, then applies the original token-level indexer only within the retained candidate blocks, preserving the same top-sparse pattern for downstream attention. Benchmarks show HISA achieves significant speedups at 64K context and matches the quality of DeepSeek-V3.2 and GLM-5 without additional training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
2d ago

Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attention

arXiv:2610.42147v1 Announce Type: new Abstract: Agentic workflows are increasingly shifting towards long-horizon tasks, such as open-ended scientific work, where models benefit from long context wind...

By Ales Kubicek, Andrea Caciolai, Liangzu Peng, Sukrit Sumant, Benjamin Hoover, Fraser Greenlee, Ruairidh M. Battleday, Radim Urban, Torsten Hoefler, James C. R. Whittington
agentsdiffusionefficiency
More like this →
arXiv AI
Oct 26

HyScale: Scaling Long-Context Training for Learned Sparse Attention

HyScale is a long‑context training system designed to accelerate learned sparse attention for million‑token contexts. It introduces HyTopK, which computes each query’s exact top‑k indices on the fly without a full score table, and selection replay, which reuses forward‑pass selections during recomputation. These techniques, combined with system‑wide optimizations, achieve a 2.07× speedup for GLM‑5.1 training on 1M‑token sequences compared to Megatron‑LM, and similar gains for DeepSeek‑V4‑Flash and Qwen3.8‑Flash‑Next on large GPU clusters.

By Yangsheng Deng, Xinjing Huang, Yulong Wang, Yunzhuo Liu, Jun Hu, Shuyong Tang, Junchi Zeng, Mingji Han, Ray Ying, Key Zhang, Luo Mai
More like this →
arXiv AI
Oct 21

Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity

The paper introduces Stratified Sparse Attention (SSA), a framework that blends deterministic top‑k retrieval with score‑guided sampling to reduce the gap between sparse and dense attention in large language models. SSA attends to high‑scoring tokens deterministically while sampling the remaining attention mass using proxy scores, thereby recovering more of the attention distribution. Experiments on benchmarks such as RULER, LongBench, LOFT, and AIME show that SSA outperforms existing sparse methods across models ranging from 3B to 27B parameters, achieving significant accuracy gains and up to 2.96× decode speedup with optimized CUDA kernels.

By Sahil Joshi, Prithvi Dixit, Aryan Katiyar, Agniva Chowdhury, Joseph E. Gonzalez, Anshumali Shrivastava, Aditya Desai, Kumar Krishna Agrawal
llmsefficiency
More like this →