arXiv AI By An Zhao, Yixiang Zhou, Zejian Li, Yi Guo, Pei Chen, Lingyun Sun

Trajectory-Regularized Distribution Distillation for One-Step Generation

Read the original on arXiv AI →

The paper introduces Trajectory-Regularized Distribution Distillation (TRD), a method that merges distribution-level and trajectory-level supervision for one-step generative models. TRD uses a momentum-mixture policy derived from a single student forward pass, eliminating the need for an extra network to estimate the student’s score. Experiments on SANA-1.6B and Stable Diffusion 3.5 demonstrate that TRD surpasses existing consistency and distribution matching baselines in generation quality and training efficiency, though it slightly reduces sample diversity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Nov 5

VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

The paper introduces Velocity Adversarial Distillation (VAD), a method that uses a learned critic to compare velocity fields of student and teacher diffusion models, turning disagreements into learning signals. VAD unifies several existing distillation techniques and leads to VA‑TDM, which applies trajectory distribution matching and transition‑aware online velocity learning to supervise partial denoising along the student’s sampling path. Experiments demonstrate that VA‑TDM achieves state‑of‑the‑art performance for four‑step text‑to‑image and text‑to‑video generation, outperforming prior few‑step methods and approaching the quality of many‑step baselines.

By Yihong Luo, Tianyu Pang, Tianyang Hu
arXiv Machine Learning
Oct 8

Consistent Distribution Matching for Data-Free Diffusion Distillation

Consistent Distribution Matching is a data‑free distillation technique that accelerates diffusion and flow models by unifying sample generation and score estimation in a single student network. The method requires only a frozen teacher and a trainable student, optimizing a single objective that provably leads to Wasserstein convergence of the student’s flow‑map to the teacher’s marginals. On ImageNet 256×256, it achieves an FID of 2.04 with one function evaluation and 1.37 with four evaluations after just 40 training epochs, outperforming existing data‑free baselines.

By Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao