← Back to all news
arXiv Machine Learning By Zongze Wu (Adobe), Xianbang Wang (MIT), Zhengyang Geng (Adobe), Mingdeng Cao (Adobe), Zhoutong Zhang (Adobe), Richard Zhang (Adobe), Eli Shechtman (Adobe), Ruiming Cao (Adobe)

A Better Critic Matters for Distribution Matching Distillation

Read the original on arXiv Machine Learning →

The paper investigates how the critic in Distribution Matching Distillation (DMD) should be designed to provide a reliable training signal. Experiments on ImageNet with seven SiT model sizes show that keeping a small critic while increasing teacher size can degrade student quality and cause training collapse; scaling the critic together with the teacher (joint scaling) prevents this collapse and lets students benefit from larger teachers. Additionally, the authors propose a step‑aware critic (STAC) that conditions on the student’s generation step, and demonstrate that joint scaling combined with STAC consistently improves few‑step distillation for class‑conditional ImageNet generation, text‑to‑image generation with SDXL and FLUX.2, and text‑to‑video generation with Wan2.1.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 8

Consistent Distribution Matching for Data-Free Diffusion Distillation

Consistent Distribution Matching is a data‑free distillation technique that accelerates diffusion and flow models by unifying sample generation and score estimation in a single student network. The method requires only a frozen teacher and a trainable student, optimizing a single objective that provably leads to Wasserstein convergence of the student’s flow‑map to the teacher’s marginals. On ImageNet 256×256, it achieves an FID of 2.04 with one function evaluation and 1.37 with four evaluations after just 40 training epochs, outperforming existing data‑free baselines.

By Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao
diffusionefficiencybenchmarks
More like this →
arXiv Computer Vision
Oct 5

VDOT++: Unified Few-Step Video Generation via Unbalanced Optimal Transport Distillation

The paper introduces VDOT++, a unified distillation framework for few-step video generation across text-to-video, image-to-video, and condition-based tasks. It extends the previous VDOT method by incorporating an asymmetric unbalanced optimal transport distillation that allows student tokens to carry less mass, and an α1 ground cost that uses a weighted median to preserve modes. The approach combines distribution matching with adversarial refinement and cross-scale distillation, achieving competitive performance on multiple video benchmarks with only four sampling steps.

By Yutong Wang, Xingtong Ge, Enhuai Liu, Yunke Wang, Tianfan Xue, Yu Qiao, Yaohui Wang, Xinyuan Chen, Chang Xu
diffusionefficiencybenchmarkssafety
More like this →