arXiv:2605. 30116v2 Announce Type: replace-cross Abstract: Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models.
The paper introduces VDOT++, a unified distillation framework for few-step video generation across text-to-video, image-to-video, and condition-based tasks. It extends the previous VDOT method by incorporating an asymmetric unbalanced optimal transport distillation that allows student tokens to carry less mass, and an α1 ground cost that uses a weighted median to preserve modes. The approach combines distribution matching with adversarial refinement and cross-scale distillation, achieving competitive performance on multiple video benchmarks with only four sampling steps.
DMAD (Distribution Matching as Adversarial Distillation) reinterprets distribution matching as a classification task, enabling a few‑step student model to learn log‑density ratios directly via two discriminator heads on a shared backbone. This eliminates the need for an auxiliary diffusion model, reducing memory and computation overhead. Experiments show DMAD achieves state‑of‑the‑art Fréchet Inception Distance scores on ImageNet‑64x64, COCO‑10K, and VBench, and outperforms competing few‑step methods in joint audio‑video generation on MiniMax‑H3‑33B.
Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamic...
Consistent Distribution Matching is a data‑free distillation technique that accelerates diffusion and flow models by unifying sample generation and score estimation in a single student network. The method requires only a frozen teacher and a trainable student, optimizing a single objective that provably leads to Wasserstein convergence of the student’s flow‑map to the teacher’s marginals. On ImageNet 256×256, it achieves an FID of 2.04 with one function evaluation and 1.37 with four evaluations after just 40 training epochs, outperforming existing data‑free baselines.
arXiv:2608. 09226v1 Announce Type: cross Abstract: Efficient text-to-image generation requires both reinforcement-learning (RL)-based reward alignment and few-step distillation, yet these procedures are typically performed sequentially, increasing training cost and risking the loss of reward gains during compression.
arXiv:2610.26025v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) deliver high-fidelity image and video generation, but inference requires many sequential model evaluations. Two...
The paper introduces TACD, a Terminal‑Amplification‑Controlled Distillation method that trains efficient text‑to‑motion generators using on‑policy flow distillation from pretrained teachers, without requiring real‑motion data. TACD addresses a failure mode where velocity matching over‑weights errors near the denoising endpoint by tying the latest teacher query to the student’s step size, thereby bounding loss weights in clean‑motion space. Experiments on HumanML3D, KIT‑ML, HY‑Motion, and Kimodo show that TACD improves few‑step generation quality, reduces FID scores, and achieves significant speedups and memory savings compared to teacher models.
The paper investigates how the critic in Distribution Matching Distillation (DMD) should be designed to provide a reliable training signal. Experiments on ImageNet with seven SiT model sizes show that keeping a small critic while increasing teacher size can degrade student quality and cause training collapse; scaling the critic together with the teacher (joint scaling) prevents this collapse and lets students benefit from larger teachers. Additionally, the authors propose a step‑aware critic (STAC) that conditions on the student’s generation step, and demonstrate that joint scaling combined with STAC consistently improves few‑step distillation for class‑conditional ImageNet generation, text‑to‑image generation with SDXL and FLUX.2, and text‑to‑video generation with Wan2.1.
Efficient text-to-image generation requires both reinforcement-learning (RL)-based reward alignment and few-step distillation, yet these procedures are typically performed sequentially, increasing training cost and risking the loss of reward gains during compression. We instead take an RL-native perspective: diffusion RL already generates reward-scored finite-step trajectories, whose intermediate states provide a natural source of distillation supervision rather than a disposable byproduct of sampling.
arXiv:2606. 29287v1 Announce Type: new Abstract: Diffusion and continuous-flow generative models achieve high-quality generation, and their deterministic sampling can be formulated as solving learned ODE dynamics.
arXiv:2609.40037v1 Announce Type: new Abstract: Few-step autoregressive video generation enables efficient streaming synthesis, but errors introduced in early temporal blocks are reused as context an...