Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost. Although many acceleration methods have been proposed, a central challenge is that the most effective acceleration strategy is highly instance-specific: a recipe that works well for one combination of model, hardware, and inference configuration often does not transfer to another.
arXiv:2606. 23743v1 Announce Type: cross Abstract: Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost.
arXiv:2607. 13770v1 Announce Type: cross Abstract: Video diffusion transformers (vDiTs) generate high quality video but introduce extremely high compute cost due to the long diffusion timesteps and self attention computation.
arXiv:2609.38413v1 Announce Type: new Abstract: Vision-language models (VLMs) can answer questions about hour-long videos, but processing every frame is prohibitively expensive, even though the evide...
arXiv:2508.15774v2 Announce Type: replace Abstract: Visual diffusion models achieve remarkable progress, yet they are typically trained at limited resolutions due to the lack of high-resolution data...
arXiv:2607. 27380v2 Announce Type: replace-cross Abstract: Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt.
arXiv:2609.23153v1 Announce Type: new Abstract: Video diffusion transformers are expensive because attention dominates long spatiotemporal token sequences. We identify the \emph{high-sparsity trap}:...
arXiv:2605. 31603v2 Announce Type: replace-cross Abstract: Connector-based video unified models have demonstrated strong capability in instruction-grounded video synthesis, but integrating a large high-fidelity generator into the unified training loop is computationally prohibitive, limiting achievable visual quality.
arXiv:2607. 20125v1 Announce Type: cross Abstract: Autoregressive (AR) video diffusion models have become a promising paradigm for long and streaming video synthesis, but the continuously growing Key-Value (KV) cache makes attention the dominant inference cost, especially at high resolution where each frame contributes many tokens.
The paper introduces vidax, an open‑source JAX/Flax inference engine that enables video generative models to run on accelerator meshes such as Cloud TPU pods. Vidax includes a zero‑copy PyTorch‑to‑JAX weight translator and supports a wide range of spatiotemporal architectures—including Diffusion Transformers, Mixture‑of‑Transformers, 3D VAEs, and text encoders—without requiring PyTorch in the execution path. The framework unifies 1D tensor parallelism with DeepSpeed‑Ulysses sequence parallelism on a single JAX sharding mesh, incorporates TPU flash‑attention kernels, and implements per‑layer weight offloading to handle reference resolutions that exceed single‑device memory, with benchmarks on TPU v4‑8 hardware and documentation of real‑world numerical bugs.
arXiv:2510. 09608v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) could power real-time assistants and autonomous agents, but they face a critical challenge: understanding near-infinite video streams without escalating latency and memory usage.
The paper introduces TRACK, a training‑free trajectory routing method that accelerates video diffusion by selectively switching between large and small models during denoising steps. A calibration process generates a disagreement score map, guiding the selection of the appropriate model at each step to maintain quality while reducing computational cost. Experiments on Wan 2.1, Cosmos 3, TurboDiffusion, and FastVideo show speedups ranging from 1.95× to 2.73× with comparable quality and diversity.
arXiv:2609.37001v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the compu...