arXiv:2610.08772v1 Announce Type: new
Abstract: Diffusion Transformers (DiTs) have achieved strong performance in image and video generation, but the quadratic complexity of full attention makes high...
By Liao Ma, Jiayi Song, Yunfeng Wu, Songhua Liu, Peilin Zhao
arXiv:2609.37001v1 Announce Type: cross
Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...
By Xingyu Jia, Baole Ai, Ang Wang, Kang Zhao, Yong Li
arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.
By Enhuai Liu, Yunke Wang, Yutong Wang, Changming Sun, Chang Xu
arXiv:2610.29441v1 Announce Type: new
Abstract: Full 3D attention is costly in video Diffusion Transformers. Block-sparse attention reduces this cost by computing only selected attention blocks. Howe...
By Tianze Xia, Xiaoling Zhou, Ruya Lan, Yiqiao Liao, Chengru Song, Shiming Xiang
The paper introduces Growing Sparsity and Reduced Search (GSRS), a training‑free framework that accelerates Diffusion Transformers for video generation by progressively increasing sparsity and reusing early attention blocks to limit search space. GSRS uses Top‑p selection for adaptive sparsity and a precise evaluation scheme that computes exact attention weights with negligible overhead. Experiments on HunyuanVideo and Wan 2.1 show 1.55×–2.21× speedups while preserving generation quality comparable to full attention.
By Yun Dai, Yuyun Chen, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng
arXiv:2609.15810v1 Announce Type: new
Abstract: Diffusion Transformers deliver state-of-the-art video generation, but their long spatiotemporal sequences make attention the dominant deployment cost,...
By Xingyang Li, Dongyun Zou, Shining Zhang, Jiacheng Chen, Haocheng Xi, Lvmin Zhang, Jun-Yan Zhu, Song Han, Zhekai Zhang, Yujun Lin, Muyang Li
arXiv:2606. 23743v1 Announce Type: cross Abstract: Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost.
By Yitong Li, Junsong Chen, Haopeng Li, Haozhe Liu, Jincheng Yu, Ligeng Zhu, Ping Luo, Song Han, Enze Xie
TriTD introduces a tri-partite distillation framework for autoregressive video diffusion, addressing three error sources—base model error, train‑inference distribution mismatch, and upstream error propagation. It combines Next‑Node Trajectory Distillation for efficient ODE initialization, Chunk‑Graded Noising to reduce noise mismatch and eliminate redundant KV‑cache updates, and Orthogonal‑Parallel DMD to enforce distribution matching while suppressing off‑axis drift. Experiments demonstrate that TriTD outperforms existing baselines in visual quality and achieves real‑time throughput of 26.8 FPS on a single H100 GPU.
By Jia Li, Xurui Peng, Haowei Zhu, Tianyu Zhao, Jiexi Wang, Youwei Zheng, Yuxi Ren, Xinglong Wu, Xing Wang, Hayden Kwok-Hay So
arXiv:2610.35706v1 Announce Type: cross
Abstract: Attention over tens of thousands of visual tokens dominates the inference of video diffusion transformers (DiTs). Existing methods reduce this cost t...
By Yeonjae Kim, Omin Kwon, Seong Hoon Seo, Jae W. Lee
Sink‑Sparse Hybrid Attention is a train‑free method that replaces dense self‑attention in Diffusion Transformers with reusable static sink masks and a lightweight pooled‑KV branch, keeping model weights and sampling schedules unchanged. It identifies reusable high‑mass structures from a small offline calibration set and applies them across future generations. Experiments on Wan2.1‑T2V‑1.3B and Z‑Image‑Turbo show 1.54× and 1.57× end‑to‑end speedups while preserving generation quality.
By Bingxin Xing, Yu Zhang, Jiatong Ding, Lei Lu, Ping Wang
PISA introduces a training‑free Piecewise Sparse Attention mechanism that preserves full attention span while reducing complexity. It keeps exact computation for critical blocks and approximates non‑critical ones using block‑wise Taylor expansion, avoiding the quality loss seen in conventional sparse attention. Experiments show significant speedups—up to 2.57× on Hunyuan‑Video—while maintaining or improving visual quality across video and image generation tasks.
By Haopeng Li, Shitong Shao, Wenliang Zhong, Zikai Zhou, Lichen Bai, Hui Xiong, Zeke Xie
arXiv:2609.32540v2 Announce Type: replace-cross
Abstract: Few-step autoregressive video diffusion generates a long video by splitting the video into temporal chunks and generating chunk-by-chunk, eac...
By Yikai Wang, Xiao Han, Mengmeng Xu, Juan Camilo Perez, Yiannis Douratsos, Sen He, Zijian Zhou, Fei Zhang, Zhaochong An, Juan-Manuel Perez-Rua, Chen Change Loy, Tao Xiang