arXiv Machine Learning November 2, 2026 By Jiedong Zhuang, Peng Zhao, Jialin Tang, Haoji Hu

SparseGDN: Chunk Sparse in Linear Attention

Read the original on arXiv Machine Learning →

SparseGDN is a hardware‑aware framework that makes Gated Delta Networks (GDNs) more efficient for long‑context sequence modeling. It combines structured intra‑chunk approximation with an input‑dependent recurrent‑event selector, producing outputs for every token while reducing dense computation. The GPU implementation updates only selected events and reconstructs token outputs from compact state snapshots, cutting computation and memory traffic.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Oct 12

DeltaLLM: Delta-Key Sparse Attention for Edge LLMs

DeltaLLM is a training‑free framework that accelerates on‑device large language model inference by introducing delta‑key sparse attention and a vectorized structured sparse online softmax. It reuses delta keys across query heads, preserves exact attention near the diagonal while approximating non‑local regions, and exploits hardware‑friendly sparsity with a specialized vector‑processor architecture. Experiments on BitNet‑b1.58‑2B‑4T and LLaMA3.2‑1B‑Instruct show up to 60% delta‑key sparsity, 63% structured attention‑score sparsity, and a 1.62× speedup on the KV260 FPGA with minimal accuracy loss.

By Jiawen Qi, Jiayu Liu, Chang Gao, Zhaochun Ren, Qinyu Chen

llms fine-tuning efficiency

arXiv Machine Learning

Sep 1

Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware

arXiv:2608.30439v1 Announce Type: cross Abstract: Inference with transformer-based large language models (LLMs) is often limited by the memory-bound KV cache and quadratic attention cost. State-space...

By Simon Richter, Ruhai Lin, Jason Yik, Taylor Kergan, Rui-Jie Zhu, Farshad Moradi, Jason Eshraghian

llms efficiency

arXiv Machine Learning

Aug 18

DeltaLog: Deferred Materialization of Recurrent States for Linear Attention Decoding

arXiv:2608. 15533v1 Announce Type: cross Abstract: Linear attention models eliminate the quadratic prefix computation and context-growing KV cache of softmax attention by replacing pairwise token interactions with recurrent state updates.

By Junqing Lin, Jingwei Sun, Guangzhong Sun

efficiency

arXiv AI

Sep 30

LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization

arXiv:2609.38166v1 Announce Type: cross Abstract: Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Atte...

By Yi Pan, Haocheng Xi, Kan Zhu, Xingyang Li, Yibo Wu, Mayank Mishra, Hongtao Zhang, William X. Zheng, Baris Kasikci, Song Han, Kurt Keutzer, Rishabh Iyer, Ion Stoica

llms efficiency

arXiv AI

Jun 2

HASTE: Hardware-Aware Dynamic Sparse Training for Large Output Spaces

arXiv:2606. 01117v1 Announce Type: cross Abstract: Extreme multi-label classification (XMC) involves learning models over large output spaces with millions of labels, making the output layer a memory-compute bottleneck.

By Nasib Ullah, Jinbin Zhang, Jean Lucien Randrianantenaina, Erik Schultheis, Rohit Babbar

efficiency benchmarks safety

arXiv Computation and Language

Aug 21

FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving

arXiv:2608. 19758v1 Announce Type: new Abstract: Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, particularly during the compute-intensive prefilling phase.

By Qihang Fan, Huaibo Huang, Zhiying Wu, Bingning Wang, Ran He

llms efficiency

One email a morning, machine-written

Related stories

DeltaLLM: Delta-Key Sparse Attention for Edge LLMs

Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware

DeltaLog: Deferred Materialization of Recurrent States for Linear Attention Decoding

LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization

HASTE: Hardware-Aware Dynamic Sparse Training for Large Output Spaces

FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving