arXiv AI By Pardis Taghavi, Reza Langari, Gaurav Pandey

Partition the Support, Reconstruct the Residual: Training-Free Sparse Attention for Video Generation and World Models

Read the original on arXiv AI →

The paper introduces SparsePR, a training‑free block‑sparse attention method for video transformers that partitions query‑key responses and reconstructs the residual via probe‑fitted affine corrections. By pairing sampled‑query key responses into K/V groups and using centroids to guide shared routing, SparsePR reduces attention‑reconstruction error across diverse video generation and world‑model tasks. Experiments show consistent error reductions, with probe fitting contributing most of the improvement, while maintaining generation quality at 22.0–26.0% executed‑pair density and delivering 1.48×–2.61× speedups.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Nov 5

DecoupAttn: Decoupled Sparse Attention for Training-Free Acceleration of Video Diffusion

DecoupAttn introduces a decoupled sparse attention mechanism for video diffusion transformers, separating key block computation from error compensation. It uses a distortion‑budget objective to allocate exact attention across timesteps, layers, and heads, and a reassign cache to refresh key assignments while keeping budgeted blocks fixed. The method is training‑free, requiring only a few calibration clips to compile an offline plan, after which inference runs with standard block‑sparse kernels and a fused compensation kernel.

By Yanjin Li, Bingbing Qiu, Qiang Huang, Rui Qian, Yanfeng Li, Jian Wang, Xiaoying Tang, Mingda Zhang
Hugging Face Trending Papers
Aug 13

SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention

Diffusion Transformers (DiTs) incur quadratic self-attention cost over spatiotemporal tokens. Existing training-free sparse attention methods often construct sparse masks from block-level or cluster-level proxy scores, which can obscure fine-grained differences among keys and miss high contribution keys under aggressive sparsity.