arXiv:2603.17825v2 Announce Type: replace
Abstract: In this work, we study the role of Massive Activations (MAs), which are rare, high-magnitude spikes confined to a few fixed hidden dimensions in vi...
By Xianhang Cheng, Yujian Zheng, Zhenyu Xie, Tingting Liao, Hao Li
arXiv:2605. 14513v2 Announce Type: replace-cross Abstract: Sparse attention accelerates video diffusion by allowing each attention head to focus on only a small subset of interactions.
By Xuzhe Zheng, Yuexiao Ma, Jing Xu, Xiawu Zheng, Rongrong Ji, Fei Chao
Focused Forcing is a training‑free method for selecting key‑value (KV) cache entries in autoregressive video diffusion models. It selects historical frames per generated frame by combining attention and diversity scores, and allocates larger budgets to more important heads. The approach yields up to 1.48× faster generation while improving visual quality and text alignment across several generation paradigms.
By Peiliang Cai, Evelyn Zhang, Jiacheng Liu, Hao Lin, Ruiqi Zhang, Weile Mo, Yue Ma, Shikang Zheng, Jiehang Huang, Dongrui Liu, Linfeng Zhang
arXiv:2609.08505v1 Announce Type: cross
Abstract: Reliable video generation requires more than high-quality frames to form a coherent story: a model must maintain a persistent state, transporting vis...
By Luyao Tang, Bingjun Luo, Dong Yi, Jialin Guo, Haoning Xi, Cheng Chen, Yizhou Yu, Chaoqi Chen
arXiv:2601. 11641v3 Announce Type: replace-cross Abstract: While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent to self-attention mechanisms, creating significant barriers to practical deployment.
By Yuxi Liu, Yipeng Hu, Zekun Zhang, Kunze Jiang, Kun Yuan
arXiv:2610.29441v1 Announce Type: new
Abstract: Full 3D attention is costly in video Diffusion Transformers. Block-sparse attention reduces this cost by computing only selected attention blocks. Howe...
By Tianze Xia, Xiaoling Zhou, Ruya Lan, Yiqiao Liao, Chengru Song, Shiming Xiang
arXiv:2609.23658v1 Announce Type: cross
Abstract: Despite impressive visual quality, state-of-the-art video diffusion models often generate content that violates real-world physical laws. While exist...
By Yueyan Li, Haibo Wang, Caixia Yuan, Xiaojie Wang
The paper introduces Growing Sparsity and Reduced Search (GSRS), a training‑free framework that accelerates Diffusion Transformers for video generation by progressively increasing sparsity and reusing early attention blocks to limit search space. GSRS uses Top‑p selection for adaptive sparsity and a precise evaluation scheme that computes exact attention weights with negligible overhead. Experiments on HunyuanVideo and Wan 2.1 show 1.55×–2.21× speedups while preserving generation quality comparable to full attention.
By Yun Dai, Yuyun Chen, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng
arXiv:2609.37001v1 Announce Type: cross
Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...
By Xingyu Jia, Baole Ai, Ang Wang, Kang Zhao, Yong Li
arXiv:2602. 01801v2 Announce Type: replace-cross Abstract: Autoregressive video diffusion models enable streaming generation, opening the door to long-form synthesis, video world models, and interactive neural game engines.
By Dvir Samuel, Issar Tzachor, Matan Levy, Michael Green, Gal Chechik, Rami Ben-Ari
The paper introduces SalTempto, a new video saliency benchmark featuring 224 highly dynamic clips from HACS‑Segments, each with event context and gaze data from up to 16 subjects. It shows that static baselines perform poorly on SalTempto compared to LEDOV, while fine‑tuned temporal models achieve significant gains, yet still leave a large portion of explainable gaze information unaccounted for. The dataset highlights the need for better temporal modeling in video saliency research.
By Susmit Agrawal, Rebecca Wanner, Juliane Verwiebe, Matthias Tangemann, Matthias Bethge, Matthias K\"ummerer
VMonarch introduces a structured Monarch matrix to replace the quadratic attention in Video Diffusion Transformers, enabling sub‑quadratic computation. The method captures intra‑ and inter‑frame correlations, mitigates instability artifacts, and integrates an online entropy algorithm with FlashAttention for efficient updates. Experiments show comparable or better generation quality, a 17.5× reduction in FLOPs, and over 5× speedup for long videos, outperforming state‑of‑the‑art sparse attention at 90% sparsity.
By Cheng Liang, Haoxian Chen, Liang Hou, Qi Fan, Gangshan Wu, Xin Tao, Limin Wang