arXiv Computer Vision
Backend-Agnostic Sparse Attention for Fast High-Resolution Visual Generation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Sep 30
Parameterized Stripe Attention for Efficient Video Generation
arXiv:2609.37001v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...
arXiv Computer Vision
Sep 22
SparkDiffusion: Mitigating the High-Sparsity Trap --- A Unified Framework for up to $265\times$ Single-GPU Acceleration of Visual Generation
arXiv:2609.23153v1 Announce Type: new Abstract: Video diffusion transformers are expensive because attention dominates long spatiotemporal token sequences. We identify the \emph{high-sparsity trap}:...
arXiv AI
Jun 24
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
arXiv:2606. 23743v1 Announce Type: cross Abstract: Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost.
arXiv Machine Learning
Jul 2
Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
arXiv:2601. 11641v3 Announce Type: replace-cross Abstract: While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent to self-attention mechanisms, creating significant barriers to practical deployment.
arXiv Computer Vision
Sep 1
Fast and Accurate Image Restoration and Generation with Rank Enhanced Linear Attention
arXiv:2505.16157v3 Announce Type: replace Abstract: Transformer-based models have made remarkable progress in image restoration (IR) tasks. However, the quadratic complexity of self-attention in Tran...
arXiv Computer Vision
Sep 18
Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation
The paper investigates how attention sparsity behaves in autoregressive image generation, finding a distinct diagonal sparsity pattern due to spatial locality of visual tokens. It introduces a diagonal‑aware sparse attention mechanism that skips KV entries along the diagonal within a recent window, achieving up to 3.1× higher throughput and 1.19× lower latency with less than 2% quality loss compared to dense inference.