arXiv Computer Vision
DecoupAttn: Decoupled Sparse Attention for Training-Free Acceleration of Video Diffusion
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Oct 21
PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers
PISA introduces a training‑free Piecewise Sparse Attention mechanism that preserves full attention span while reducing complexity. It keeps exact computation for critical blocks and approximates non‑critical ones using block‑wise Taylor expansion, avoiding the quality loss seen in conventional sparse attention. Experiments show significant speedups—up to 2.57× on Hunyuan‑Video—while maintaining or improving visual quality across video and image generation tasks.
arXiv AI
Aug 11
HEART: Exploiting Head Heterogeneity in Sparse Attention for Video Diffusion
arXiv:2605. 14513v2 Announce Type: replace-cross Abstract: Sparse attention accelerates video diffusion by allowing each attention head to focus on only a small subset of interactions.
arXiv Computer Vision
Oct 9
iCATS: Fast Video Generation via Interaction-Aware Sparse Attention and Timestep-Adaptive Sparsity
iCATS introduces a training‑free sparse attention mechanism for Diffusion Transformers that accelerates video generation by reducing computations. It improves importance estimation by clustering query‑key dot‑product interactions and reformulates the objective as a quadratic form for low‑cost computation. The method also adapts sparsity dynamically across denoising timesteps using an SNR‑guided schedule and employs a tail‑merging strategy to enhance GPU kernel utilization, achieving significant speedups while maintaining high PSNR scores.
arXiv AI
Sep 30
Parameterized Stripe Attention for Efficient Video Generation
arXiv:2609.37001v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...
arXiv AI
Aug 13
LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration
arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.
arXiv Computer Vision
Oct 22
Sparsity Should Be ALLOTted by Output: Training-Free Allocation of Block-Sparse Attention for Video Diffusion Transformers
arXiv:2610.29441v1 Announce Type: new Abstract: Full 3D attention is costly in video Diffusion Transformers. Block-sparse attention reduces this cost by computing only selected attention blocks. Howe...