The paper introduces vidax, an open‑source JAX/Flax inference engine that enables video generative models to run on accelerator meshes such as Cloud TPU pods. Vidax includes a zero‑copy PyTorch‑to‑JAX weight translator and supports a wide range of spatiotemporal architectures—including Diffusion Transformers, Mixture‑of‑Transformers, 3D VAEs, and text encoders—without requiring PyTorch in the execution path. The framework unifies 1D tensor parallelism with DeepSpeed‑Ulysses sequence parallelism on a single JAX sharding mesh, incorporates TPU flash‑attention kernels, and implements per‑layer weight offloading to handle reference resolutions that exceed single‑device memory, with benchmarks on TPU v4‑8 hardware and documentation of real‑world numerical bugs.
arXiv:2606. 23743v1 Announce Type: cross Abstract: Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost.
Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost. Although many acceleration methods have been proposed, a central challenge is that the most effective acceleration strategy is highly instance-specific: a recipe that works well for one combination of model, hardware, and inference configuration often does not transfer to another.
arXiv:2607.14088v2 Announce Type: replace Abstract: Video generation models typically rely on 3D-VAEs trained for pixel-level reconstruction, whose latent spaces may underrepresent semantic structure...
arXiv:2605.30263v2 Announce Type: replace Abstract: Recent video diffusion foundation models have achieved remarkable progress in high-quality video generation, yet turning them into real-time intera...
arXiv:2609.10355v1 Announce Type: cross Abstract: Video understanding has rapidly evolved toward video large language models (VideoLLMs): systems that couple video representations with pretrained lar...
arXiv:2606. 02800v1 Announce Type: cross Abstract: We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture.
arXiv:2605. 31603v2 Announce Type: replace-cross Abstract: Connector-based video unified models have demonstrated strong capability in instruction-grounded video synthesis, but integrating a large high-fidelity generator into the unified training loop is computationally prohibitive, limiting achievable visual quality.
arXiv:2609.23153v1 Announce Type: new Abstract: Video diffusion transformers are expensive because attention dominates long spatiotemporal token sequences. We identify the \emph{high-sparsity trap}:...
arXiv:2607. 22696v1 Announce Type: cross Abstract: High-resolution video diffusion models built on Diffusion Transformers (DiTs) deliver strong fidelity but quickly exhaust the memory budget of a single workstation.
arXiv:2610.00686v1 Announce Type: new Abstract: Recent video-based world models pair the scalability of autoregressive (AR) prediction with the visual quality of diffusion models. The choice of scene...
arXiv:2506. 10915v2 Announce Type: replace-cross Abstract: Text-to-video generation has significantly enriched content creation and holds the potential to evolve into powerful world simulators.
arXiv:2607. 18367v1 Announce Type: new Abstract: Unlike conventional video game development, which relies on labor-intensive pipelines for asset production, animation, physics, and programming, video world models generate interactive environments from user inputs instantly.