arXiv AI

VMonarch: Efficient Video Diffusion Transformers with Structured Attention

VMonarch introduces a structured Monarch matrix to replace the quadratic attention in Video Diffusion Transformers, enabling sub‑quadratic computation. The method captures intra‑ and inter‑frame correlations, mitigates instability artifacts, and integrates an online entropy algorithm with FlashAttention for efficient updates. Experiments show comparable or better generation quality, a 17.5× reduction in FLOPs, and over 5× speedup for long videos, outperforming state‑of‑the‑art sparse attention at 90% sparsity.

arXiv Computer Vision
Oct 16

Sparse Attention for Video Generation Acceleration via Growing Sparsity and Reduced Search

The paper introduces Growing Sparsity and Reduced Search (GSRS), a training‑free framework that accelerates Diffusion Transformers for video generation by progressively increasing sparsity and reusing early attention blocks to limit search space. GSRS uses Top‑p selection for adaptive sparsity and a precise evaluation scheme that computes exact attention weights with negligible overhead. Experiments on HunyuanVideo and Wan 2.1 show 1.55×–2.21× speedups while preserving generation quality comparable to full attention.

By Yun Dai, Yuyun Chen, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng
arXiv Computer Vision
Oct 21

PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers

PISA introduces a training‑free Piecewise Sparse Attention mechanism that preserves full attention span while reducing complexity. It keeps exact computation for critical blocks and approximates non‑critical ones using block‑wise Taylor expansion, avoiding the quality loss seen in conventional sparse attention. Experiments show significant speedups—up to 2.57× on Hunyuan‑Video—while maintaining or improving visual quality across video and image generation tasks.

By Haopeng Li, Shitong Shao, Wenliang Zhong, Zikai Zhou, Lichen Bai, Hui Xiong, Zeke Xie
arXiv AI
Jul 16

Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlations

arXiv:2607. 13770v1 Announce Type: cross Abstract: Video diffusion transformers (vDiTs) generate high quality video but introduce extremely high compute cost due to the long diffusion timesteps and self attention computation.

By Wenxuan Miao, Haosong Liu, Weiming Hu, Zihan Liu, Aiyue Chen, Jianlin Yu, Yiwu Yao, Yiming Gan, Jieru Zhao, Jingwen Leng, Minyi Guo, Yu Feng
arXiv Machine Learning
Jul 2

Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers

arXiv:2601. 11641v3 Announce Type: replace-cross Abstract: While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent to self-attention mechanisms, creating significant barriers to practical deployment.

By Yuxi Liu, Yipeng Hu, Zekun Zhang, Kunze Jiang, Kun Yuan
arXiv AI
Oct 31

Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion

Focused Forcing is a training‑free method for selecting key‑value (KV) cache entries in autoregressive video diffusion models. It selects historical frames per generated frame by combining attention and diversity scores, and allocates larger budgets to more important heads. The approach yields up to 1.48× faster generation while improving visual quality and text alignment across several generation paradigms.

By Peiliang Cai, Evelyn Zhang, Jiacheng Liu, Hao Lin, Ruiqi Zhang, Weile Mo, Yue Ma, Shikang Zheng, Jiehang Huang, Dongrui Liu, Linfeng Zhang