Hugging Face Trending Papers August 13, 2026

SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention

Read the original on Hugging Face Trending Papers →

Diffusion Transformers (DiTs) incur quadratic self-attention cost over spatiotemporal tokens. Existing training-free sparse attention methods often construct sparse masks from block-level or cluster-level proxy scores, which can obscure fine-grained differences among keys and miss high contribution keys under aggressive sparsity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Computer Vision

3d ago

PARK: Accurate Block Retrieval for Sparse Attention in Video Diffusion Transformers

arXiv:2609.38978v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) have become a dominant architecture for video generation, but their efficiency is limited by the quadratic complexity of...

By Yun Dai, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng

llms diffusion

arXiv AI

Jul 7

HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

arXiv:2607. 03012v1 Announce Type: cross Abstract: Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation.

By Dongyeun Lee, Amir Zandieh, Vahab Mirrokni, Junmo Kim, Insu Han

llms diffusion

arXiv AI

Jul 23

ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers

arXiv:2607. 20214v1 Announce Type: cross Abstract: The quadratic $N\times N$ attention score matrix remains a central obstacle to extending Transformers to longer input lengths.

By Mahdi Heidari, Mohammad Mahdi Rahimi, Jaekyun Moon

llms efficiency

arXiv AI

Aug 13

LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration

arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.

By Enhuai Liu, Yunke Wang, Yutong Wang, Changming Sun, Chang Xu

llms diffusion efficiency

arXiv AI

Aug 5

$\pi$-Attention: Online Efficient Sparse Transformers for Long-Context Modeling

arXiv:2511. 10696v3 Announce Type: replace-cross Abstract: Sparse attention is crucial in long-context Transformers, which restricts each token to a limited neighborhood and thereby reduces the quadratic cost of full self-attention.

By Pike D. Liu, Chang Liu, Yanxuan Yu

llms

arXiv Computer Vision

Sep 10

RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers

arXiv:2609.06712v2 Announce Type: replace Abstract: Diffusion Transformers (DiTs) achieve strong video generation quality, but their dense spatiotemporal self-attention scales quadratically with sequ...

By Zekun Zhang, Yixiang Cai, Yuxi Liu, Tengxu Sun, Tianle Liu, Zhoutong Wu, Haoyu Li, Baole Ai, Ang Wang, Jiamang Wang, Lin Qu, Kun Yuan

llms rag diffusion efficiency

One email a morning, machine-written

Related stories

PARK: Accurate Block Retrieval for Sparse Attention in Video Diffusion Transformers

HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers

LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration

$\pi$-Attention: Online Efficient Sparse Transformers for Long-Context Modeling

RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers