arXiv AI
6d ago

Trajectory-Regularized Distribution Distillation for One-Step Generation

The paper introduces Trajectory-Regularized Distribution Distillation (TRD), a method that merges distribution-level and trajectory-level supervision for one-step generative models. TRD uses a momentum-mixture policy derived from a single student forward pass, eliminating the need for an extra network to estimate the student’s score. Experiments on SANA-1.6B and Stable Diffusion 3.5 demonstrate that TRD surpasses existing consistency and distribution matching baselines in generation quality and training efficiency, though it slightly reduces sample diversity.

By An Zhao, Yixiang Zhou, Zejian Li, Yi Guo, Pei Chen, Lingyun Sun
arXiv Computer Vision
Nov 5

VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

The paper introduces Velocity Adversarial Distillation (VAD), a method that uses a learned critic to compare velocity fields of student and teacher diffusion models, turning disagreements into learning signals. VAD unifies several existing distillation techniques and leads to VA‑TDM, which applies trajectory distribution matching and transition‑aware online velocity learning to supervise partial denoising along the student’s sampling path. Experiments demonstrate that VA‑TDM achieves state‑of‑the‑art performance for four‑step text‑to‑image and text‑to‑video generation, outperforming prior few‑step methods and approaching the quality of many‑step baselines.

By Yihong Luo, Tianyu Pang, Tianyang Hu
arXiv Machine Learning
Jul 28

FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models

arXiv:2607. 24522v1 Announce Type: new Abstract: While on-policy distillation (OPD) effectively addresses sparse rewards and exposure bias in large language model post-training, its extension to flow models remains underexplored.

By Kaiyang Ye, Yuan Ge, Junxiang Zhang, Bei Li, Ziming Zhu, Haishu Zhao, Xiaoqian Liu, Chenglong Wang, Jingbo Zhu, Zhengtao Yu, Tong Xiao
arXiv Machine Learning
Nov 2

A Better Critic Matters for Distribution Matching Distillation

The paper investigates how the critic in Distribution Matching Distillation (DMD) should be designed to provide a reliable training signal. Experiments on ImageNet with seven SiT model sizes show that keeping a small critic while increasing teacher size can degrade student quality and cause training collapse; scaling the critic together with the teacher (joint scaling) prevents this collapse and lets students benefit from larger teachers. Additionally, the authors propose a step‑aware critic (STAC) that conditions on the student’s generation step, and demonstrate that joint scaling combined with STAC consistently improves few‑step distillation for class‑conditional ImageNet generation, text‑to‑image generation with SDXL and FLUX.2, and text‑to‑video generation with Wan2.1.

By Zongze Wu (Adobe), Xianbang Wang (MIT), Zhengyang Geng (Adobe), Mingdeng Cao (Adobe), Zhoutong Zhang (Adobe), Richard Zhang (Adobe), Eli Shechtman (Adobe), Ruiming Cao (Adobe)