The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention proposes a two-stage hierarchical indexer that replaces the flat token scan used in token-level sparse attention mechanisms like DeepSeek Sparse Attention. The method first performs block-level coarse filtering to discard irrelevant regions, then applies the original token-level indexer only within the retained candidate blocks, preserving the same top-sparse pattern for downstream attention. Benchmarks show HISA achieves significant speedups at 64K context and matches the quality of DeepSeek-V3.2 and GLM-5 without additional training.
arXiv:2610.42147v1 Announce Type: new Abstract: Agentic workflows are increasingly shifting towards long-horizon tasks, such as open-ended scientific work, where models benefit from long context wind...
arXiv:2607. 11976v1 Announce Type: new Abstract: Indexer-TopK, the operation to compute the scores and select the top-k candidates, is widely used by sparse attention kernels in large language models and vector retrieval in recommendation systems and vector databases.
HyScale is a long‑context training system designed to accelerate learned sparse attention for million‑token contexts. It introduces HyTopK, which computes each query’s exact top‑k indices on the fly without a full score table, and selection replay, which reuses forward‑pass selections during recomputation. These techniques, combined with system‑wide optimizations, achieve a 2.07× speedup for GLM‑5.1 training on 1M‑token sequences compared to Megatron‑LM, and similar gains for DeepSeek‑V4‑Flash and Qwen3.8‑Flash‑Next on large GPU clusters.
arXiv:2610.27438v1 Announce Type: cross Abstract: Sparse attention limits the keys used by each query, but finding those keys can still require a full-prefix scan at every layer. We introduce \textbf...
arXiv:2606. 09441v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) injects LLM queries with relevant documents to improve response quality.