ToDoPro introduces an early token‑reduction strategy for reference‑based image generation that uses a few denoising steps on 2× downsampled reference images to probe token importance with the existing Diffusion Transformer. Tokens are then hierarchically pruned: low‑importance tokens are discarded, mid‑importance tokens are replaced by their downsampled versions, and high‑importance tokens are retained. Experiments on three base models and two benchmarks show significant inference speedups, reduced peak GPU memory, and improved generation quality by filtering out noisy reference tokens.
arXiv:2608.21229v1 Announce Type: new Abstract: Omnimodal generation is central to a wide range of content creation and editing applications. In-context conditioning is essential to this paradigm. It...
BudgetPix introduces a compute‑adaptive tokenization framework for pixel‑space image diffusion, dynamically allocating computational resources based on visual complexity and spatial layout. It comprises an adaptive encoder using an entropy‑guided quadtree and multi‑scale patch embedder, a scale‑aware decoder, and a flexible training schedule that allows pixel‑space denoisers to operate across variable token counts. The method integrates with existing diffusion architectures, achieving comparable fidelity to larger models while using only a fraction of the compute budget, and demonstrates improved quality‑efficiency tradeoffs in both text‑to‑image and class‑conditional generation.
The paper investigates how attention sparsity behaves in autoregressive image generation, finding a distinct diagonal sparsity pattern due to spatial locality of visual tokens. It introduces a diagonal‑aware sparse attention mechanism that skips KV entries along the diagonal within a recent window, achieving up to 3.1× higher throughput and 1.19× lower latency with less than 2% quality loss compared to dense inference.
arXiv:2601. 22954v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel.
arXiv:2505. 18227v4 Announce Type: replace-cross Abstract: In Transformer architectures, tokens\textemdash discrete units derived from raw data\textemdash are formed by segmenting inputs into fixed-length chunks.
arXiv:2608. 01298v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) have emerged as a core architecture in generative modeling due to their scalability and adaptability to multimodal tasks.
arXiv:2605. 26632v2 Announce Type: replace Abstract: Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs.
The paper introduces AnchorCache, a parameter‑free token‑layout and attention‑mask design that decouples reference tokens from the target in in‑context diffusion transformers. By inserting static text anchors, the method conditions reference representations on the instruction during cache construction, enabling exact key‑value reuse across denoising steps. To restore quality lost by this structural change, the authors employ teacher‑forced velocity distillation followed by a brief on‑policy stage, achieving full‑attention quality while delivering up to 6.40× speedup in diffusion transformer inference across image, speech, and video benchmarks.
Flash-dLLM is a training‑free inference acceleration framework that improves the speed and memory efficiency of Diffusion Large Language Models (dLLMs). It tackles GPU memory I/O bottlenecks by introducing an I/O‑aware fused KV‑cache kernel and then employs a draft‑and‑verify decoding strategy that uses the dLLM itself as both drafter and verifier. Experiments on mathematical reasoning and code‑generation tasks show Flash‑dLLM outperforms existing acceleration methods, achieving up to 11.0× speedups over the Elastic‑Cache baseline.
arXiv:2605. 26632v3 Announce Type: replace Abstract: Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs.
Autoregressive image generation has emerged as a paradigm for multimodal AI systems due to its compatibility with transformer-based LLM serving infrastructures. However, generating thousands of visual...
arXiv:2502. 10389v2 Announce Type: replace-cross Abstract: Diffusion models (DMs) have become the leading choice for generative tasks across diverse domains.