arXiv Computer Vision October 9, 2026 By Chengfeng Han, Baole Ai, Xianlu Bian, Jie Yao, Zilong Huang, Ang Wang, Dandan Ding

iCATS: Fast Video Generation via Interaction-Aware Sparse Attention and Timestep-Adaptive Sparsity

Read the original on arXiv Computer Vision →

iCATS introduces a training‑free sparse attention mechanism for Diffusion Transformers that accelerates video generation by reducing computations. It improves importance estimation by clustering query‑key dot‑product interactions and reformulates the objective as a quadratic form for low‑cost computation. The method also adapts sparsity dynamically across denoising timesteps using an SNR‑guided schedule and employs a tail‑merging strategy to enhance GPU kernel utilization, achieving significant speedups while maintaining high PSNR scores.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Sep 30

Parameterized Stripe Attention for Efficient Video Generation

arXiv:2609.37001v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...

By Xingyu Jia, Baole Ai, Ang Wang, Kang Zhao, Yong Li

llms diffusion efficiency

arXiv AI

Jul 7

HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

arXiv:2607. 03012v1 Announce Type: cross Abstract: Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation.

By Dongyeun Lee, Amir Zandieh, Vahab Mirrokni, Junmo Kim, Insu Han

llms diffusion

arXiv Computer Vision

Oct 1

PARK: Accurate Block Retrieval for Sparse Attention in Video Diffusion Transformers

arXiv:2609.38978v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) have become a dominant architecture for video generation, but their efficiency is limited by the quadratic complexity of...

By Yun Dai, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng

llms diffusion

arXiv Computer Vision

Sep 15

VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention

arXiv:2609.15810v1 Announce Type: new Abstract: Diffusion Transformers deliver state-of-the-art video generation, but their long spatiotemporal sequences make attention the dominant deployment cost,...

By Xingyang Li, Dongyun Zou, Shining Zhang, Jiacheng Chen, Haocheng Xi, Lvmin Zhang, Jun-Yan Zhu, Song Han, Zhekai Zhang, Yujun Lin, Muyang Li

llms diffusion efficiency benchmarks

arXiv Machine Learning

Aug 28

ClusterAttention: A training-free speedup of bidirectional attention

ClusterAttention is a training‑free technique that speeds up bidirectional attention by recursively clustering keys and queries into fixed‑size, power‑of‑two blocks, enabling block‑sparse attention to match dense attention latency on GPUs. The method derives error bounds for sparse attention, showing tighter clusters can reduce error when compensated via centroids, and demonstrates significant speedups—up to six‑fold on large tabular data and 1.8× on video generation—while preserving over 99% of dense accuracy.

By Kasper Nordenram, Amelie Dittmann

diffusion

Hugging Face Trending Papers

Aug 13

SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention

Diffusion Transformers (DiTs) incur quadratic self-attention cost over spatiotemporal tokens. Existing training-free sparse attention methods often construct sparse masks from block-level or cluster-level proxy scores, which can obscure fine-grained differences among keys and miss high contribution keys under aggressive sparsity.