DecoupAttn: Decoupled Sparse Attention for Training-Free Acceleration of Video Diffusion
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
DecoupAttn introduces a decoupled sparse attention mechanism for video diffusion transformers, separating key block computation from error compensation. It uses a distortion‑budget objective to allocate exact attention across timesteps, layers, and heads, and a reassign cache to refresh key assignments while keeping budgeted blocks fixed. The method is training‑free, requiring only a few calibration clips to compile an offline plan, after which inference runs with standard block‑sparse kernels and a fused compensation kernel.
iCATS introduces a training‑free sparse attention mechanism for Diffusion Transformers that accelerates video generation by reducing computations. It improves importance estimation by clustering query‑key dot‑product interactions and reformulates the objective as a quadratic form for low‑cost computation. The method also adapts sparsity dynamically across denoising timesteps using an SNR‑guided schedule and employs a tail‑merging strategy to enhance GPU kernel utilization, achieving significant speedups while maintaining high PSNR scores.
Diffusion Transformers (DiTs) incur quadratic self-attention cost over spatiotemporal tokens. Existing training-free sparse attention methods often construct sparse masks from block-level or cluster-level proxy scores, which can obscure fine-grained differences among keys and miss high contribution keys under aggressive sparsity.
arXiv:2609.37001v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...
arXiv:2607. 03012v1 Announce Type: cross Abstract: Video Diffusion Transformers (VDiTs) have demonstrated significant capabilities in high-fidelity video generation.
PISA introduces a training‑free Piecewise Sparse Attention mechanism that preserves full attention span while reducing complexity. It keeps exact computation for critical blocks and approximates non‑critical ones using block‑wise Taylor expansion, avoiding the quality loss seen in conventional sparse attention. Experiments show significant speedups—up to 2.57× on Hunyuan‑Video—while maintaining or improving visual quality across video and image generation tasks.