arXiv AI
MoDDiT: Simple Token Routing in Diffusion Transformers
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
4d ago
Rethinking the design space of Vanilla Pixel Diffusion Transformers
The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.
arXiv Computer Vision
Sep 18
Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation
The paper investigates how attention sparsity behaves in autoregressive image generation, finding a distinct diagonal sparsity pattern due to spatial locality of visual tokens. It introduces a diagonal‑aware sparse attention mechanism that skips KV entries along the diagonal within a recent window, achieving up to 3.1× higher throughput and 1.19× lower latency with less than 2% quality loss compared to dense inference.
Hugging Face Trending Papers
Sep 17
Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation
Autoregressive image generation has emerged as a paradigm for multimodal AI systems due to its compatibility with transformer-based LLM serving infrastructures. However, generating thousands of visual...
arXiv Computer Vision
Sep 22
RefAM: Attention Magnets for Zero-Shot Referral Segmentation
arXiv:2509.22650v3 Announce Type: replace Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...
arXiv Computer Vision
Sep 28
Where Compute Matters: Heterogeneous Attention for Efficient Video Diffusion
The paper introduces HetA-DiT, a heterogeneous attention mechanism for video diffusion models that allocates computation based on token difficulty. A lightweight uncertainty branch predicts denoising difficulty, routing uncertain tokens through dense global attention while applying efficient local attention to reliable tokens. This adaptive routing retains global context where needed, offers a single parameter to balance quality and efficiency, and achieves competitive generation quality while only about 20% of tokens use dense attention.
arXiv Machine Learning
Jul 2
Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
arXiv:2601. 11641v3 Announce Type: replace-cross Abstract: While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent to self-attention mechanisms, creating significant barriers to practical deployment.