Hugging Face Trending Papers August 13, 2026

SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention

Read the original on Hugging Face Trending Papers →

Diffusion Transformers (DiTs) incur quadratic self-attention cost over spatiotemporal tokens. Existing training-free sparse attention methods often construct sparse masks from block-level or cluster-level proxy scores, which can obscure fine-grained differences among keys and miss high contribution keys under aggressive sparsity.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv AI

Jul 7

HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

arXiv:2607. 03012v1 Announce Type: cross Abstract: Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation.

By Dongyeun Lee, Amir Zandieh, Vahab Mirrokni, Junmo Kim, Insu Han

llms diffusion

arXiv AI

Jul 23

ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers

arXiv:2607. 20214v1 Announce Type: cross Abstract: The quadratic $N\times N$ attention score matrix remains a central obstacle to extending Transformers to longer input lengths.

By Mahdi Heidari, Mohammad Mahdi Rahimi, Jaekyun Moon

llms efficiency

arXiv AI

6d ago

LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration

arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.

By Enhuai Liu, Yunke Wang, Yutong Wang, Changming Sun, Chang Xu

llms diffusion efficiency

arXiv AI

Aug 5

$\pi$-Attention: Online Efficient Sparse Transformers for Long-Context Modeling

arXiv:2511. 10696v3 Announce Type: replace-cross Abstract: Sparse attention is crucial in long-context Transformers, which restricts each token to a limited neighborhood and thereby reduces the quadratic cost of full self-attention.

By Pike D. Liu, Chang Liu, Yanxuan Yu

llms

arXiv AI

Jun 9

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

arXiv:2605. 16928v2 Announce Type: replace-cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention.

By Yanke Zhou, Yiduo Li, Hanlin Tang, Maohua Li, Kan Liu, Tao Lan, Lin Qu, Yuan Yao, Xiaoxing Ma

llms efficiency benchmarks

arXiv Machine Learning

Jul 31

Recall Before You Rank: Similarity-Guided Top-$K$ Reuse for Efficient Long-Context Attention

arXiv:2607. 27692v1 Announce Type: cross Abstract: Top-$K$ sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries.

By Wenshuai Yao, Wenyong Zhou, Hanyong Shao, Yizhe Chen, Zhiyuan Ning, Yuannuo Feng, Ru Huang, Kechao Tang

efficiency

Related stories

HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers

LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration

$\pi$-Attention: Online Efficient Sparse Transformers for Long-Context Modeling

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

Recall Before You Rank: Similarity-Guided Top-$K$ Reuse for Efficient Long-Context Attention