arXiv Computer Vision

SparSTAR: Sparse Attention for SpaceTime AutoRegressive Video Synthesis

arXiv:2608. 10519v2 Announce Type: replace Abstract: InfinityStar extends visual autoregressive generation to video through a sequence of image and clip pyramids.

arXiv Computer Vision
Oct 21

PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers

PISA introduces a training‑free Piecewise Sparse Attention mechanism that preserves full attention span while reducing complexity. It keeps exact computation for critical blocks and approximates non‑critical ones using block‑wise Taylor expansion, avoiding the quality loss seen in conventional sparse attention. Experiments show significant speedups—up to 2.57× on Hunyuan‑Video—while maintaining or improving visual quality across video and image generation tasks.

By Haopeng Li, Shitong Shao, Wenliang Zhong, Zikai Zhou, Lichen Bai, Hui Xiong, Zeke Xie
arXiv AI
Oct 31

Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion

Focused Forcing is a training‑free method for selecting key‑value (KV) cache entries in autoregressive video diffusion models. It selects historical frames per generated frame by combining attention and diversity scores, and allocates larger budgets to more important heads. The approach yields up to 1.48× faster generation while improving visual quality and text alignment across several generation paradigms.

By Peiliang Cai, Evelyn Zhang, Jiacheng Liu, Hao Lin, Ruiqi Zhang, Weile Mo, Yue Ma, Shikang Zheng, Jiehang Huang, Dongrui Liu, Linfeng Zhang
arXiv Computer Vision
Oct 16

Sparse Attention for Video Generation Acceleration via Growing Sparsity and Reduced Search

The paper introduces Growing Sparsity and Reduced Search (GSRS), a training‑free framework that accelerates Diffusion Transformers for video generation by progressively increasing sparsity and reusing early attention blocks to limit search space. GSRS uses Top‑p selection for adaptive sparsity and a precise evaluation scheme that computes exact attention weights with negligible overhead. Experiments on HunyuanVideo and Wan 2.1 show 1.55×–2.21× speedups while preserving generation quality comparable to full attention.

By Yun Dai, Yuyun Chen, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng
arXiv Computer Vision
Sep 18

Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation

The paper investigates how attention sparsity behaves in autoregressive image generation, finding a distinct diagonal sparsity pattern due to spatial locality of visual tokens. It introduces a diagonal‑aware sparse attention mechanism that skips KV entries along the diagonal within a recent window, achieving up to 3.1× higher throughput and 1.19× lower latency with less than 2% quality loss compared to dense inference.

By Daeun Kim, Junwha Hong, Changhun Oh, Yoonsung Kim, Yoonhyeong Lee, Jongse Park
arXiv AI
Aug 20

Partition the Support, Reconstruct the Residual: Training-Free Sparse Attention for Video Generation and World Models

The paper introduces SparsePR, a training‑free block‑sparse attention method for video transformers that partitions query‑key responses and reconstructs the residual via probe‑fitted affine corrections. By pairing sampled‑query key responses into K/V groups and using centroids to guide shared routing, SparsePR reduces attention‑reconstruction error across diverse video generation and world‑model tasks. Experiments show consistent error reductions, with probe fitting contributing most of the improvement, while maintaining generation quality at 22.0–26.0% executed‑pair density and delivering 1.48×–2.61× speedups.

By Pardis Taghavi, Reza Langari, Gaurav Pandey