arXiv Machine Learning
Jun 4
SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference
arXiv:2606. 04511v1 Announce Type: cross Abstract: Sparse attention reduces compute and memory bandwidth for long-context LLM inference.
MiniMax Sparse Attention
arXiv:2606. 13392v1 Announce Type: new Abstract: Ultra-long-context capability is becoming indispensable for frontier LLMs: agentic workflows, repository-scale code reasoning, and persistent memory all require the model to jointly attend over hundreds of thousands to millions of tokens, yet the quadratic cost of softmax attention makes this untenable at deployment scale.
Related stories
arXiv Machine Learning
Jul 2
FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
arXiv:2508. 18224v3 Announce Type: replace-cross Abstract: Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language models (LLMs).
arXiv AI
Jul 22
GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding
arXiv:2605. 15250v3 Announce Type: replace-cross Abstract: Multi-head Latent Attention (MLA), the attention used in DeepSeek-V2/V3, jointly compresses keys and values into a low-rank latent and matches the H100 roofline almost perfectly.
arXiv AI
Jun 9
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
arXiv:2606. 09079v1 Announce Type: cross Abstract: Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving.
arXiv AI
Jul 1
Hierarchical Global Attention (HGA)
arXiv:2606. 30709v1 Announce Type: cross Abstract: Hierarchical Global Attention (HGA) is a drop-in replacement for dense causal attention in pretrained long-context transformers.
arXiv Machine Learning
Jul 13
COBS: Cumulant Order Block Sparse Attention
arXiv:2607. 09052v1 Announce Type: new Abstract: Block sparse attention is a hardware friendly way to alleviate the key-value (KV) cache read bottleneck in large language models (LLMs).
arXiv AI
Aug 10
Fast LapSum: Exact Differentiable Top-k at Million Scale
arXiv:2608. 06912v1 Announce Type: new Abstract: The top-$k$ operation is a fundamental building block of modern sparse computation, enabling token routing, expert activation, memory selection, and attention pruning.
arXiv AI
Jul 17
A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi
arXiv:2607. 14568v1 Announce Type: cross Abstract: A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.
arXiv AI
Jun 6
Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents
arXiv:2606. 06453v1 Announce Type: new Abstract: Sparse attention is becoming increasingly important for serving large language models (LLMs) as generation lengths continue to grow.
arXiv Machine Learning
Jun 26
PersistentKV: Page-Aware Decode Scheduling for Long-Context LLM Serving on Commodity GPUs
arXiv:2606. 26666v1 Announce Type: new Abstract: Autoregressive large language model (LLM) serving is increasingly limited by key-value (KV) cache movement rather than dense matrix multiplication.
arXiv Machine Learning
Jun 5
You Only Index Once: Cross-Layer Sparse Attention with Shared Routing
arXiv:2606. 06467v1 Announce Type: cross Abstract: Long-context inference in modern LLMs is increasingly constrained by decoding efficiency, especially in reasoning-heavy settings where models generate long intermediate chains of thought.
arXiv AI
Jun 19
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv:2606. 19348v1 Announce Type: cross Abstract: We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models -- DeepSeek-V4-Pro with 1.