arXiv AI

Geometry-Aware Time Reparameterization for Flow-Map Distillation

The paper proposes a geometry‑aware time reparameterization for flow‑map distillation, which reallocates student time to trajectory segments with high normal acceleration while preserving the teacher’s geometric paths and terminal distribution. By deriving a shared clock that equalizes a normal‑acceleration statistic and incorporating it into Lagrangian flow‑map distillation, the method improves sample quality on synthetic data, CIFAR‑10, and CelebA‑64 compared to identity‑time distillation, even in one‑step generation scenarios.

arXiv AI
6d ago

Trajectory-Regularized Distribution Distillation for One-Step Generation

The paper introduces Trajectory-Regularized Distribution Distillation (TRD), a method that merges distribution-level and trajectory-level supervision for one-step generative models. TRD uses a momentum-mixture policy derived from a single student forward pass, eliminating the need for an extra network to estimate the student’s score. Experiments on SANA-1.6B and Stable Diffusion 3.5 demonstrate that TRD surpasses existing consistency and distribution matching baselines in generation quality and training efficiency, though it slightly reduces sample diversity.

By An Zhao, Yixiang Zhou, Zejian Li, Yi Guo, Pei Chen, Lingyun Sun
arXiv AI
Oct 5

TACD: Distilling Efficient Text-to-Motion Models via Terminal Amplification Control

The paper introduces TACD, a Terminal‑Amplification‑Controlled Distillation method that trains efficient text‑to‑motion generators using on‑policy flow distillation from pretrained teachers, without requiring real‑motion data. TACD addresses a failure mode where velocity matching over‑weights errors near the denoising endpoint by tying the latest teacher query to the student’s step size, thereby bounding loss weights in clean‑motion space. Experiments on HumanML3D, KIT‑ML, HY‑Motion, and Kimodo show that TACD improves few‑step generation quality, reduces FID scores, and achieves significant speedups and memory savings compared to teacher models.

By Wei-Jin Huang, Yuan-Ming Li, Kun-Yu Lin, Wang Luo, Yinlin Zhu, Yue Yu, Shenghao Ye, Junbin Yuan, Fa-Ting Hong, Qing Zhang, Wei-Shi Zheng
arXiv Machine Learning
Oct 8

Consistent Distribution Matching for Data-Free Diffusion Distillation

Consistent Distribution Matching is a data‑free distillation technique that accelerates diffusion and flow models by unifying sample generation and score estimation in a single student network. The method requires only a frozen teacher and a trainable student, optimizing a single objective that provably leads to Wasserstein convergence of the student’s flow‑map to the teacher’s marginals. On ImageNet 256×256, it achieves an FID of 2.04 with one function evaluation and 1.37 with four evaluations after just 40 training epochs, outperforming existing data‑free baselines.

By Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao
arXiv Computer Vision
Nov 5

VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

The paper introduces Velocity Adversarial Distillation (VAD), a method that uses a learned critic to compare velocity fields of student and teacher diffusion models, turning disagreements into learning signals. VAD unifies several existing distillation techniques and leads to VA‑TDM, which applies trajectory distribution matching and transition‑aware online velocity learning to supervise partial denoising along the student’s sampling path. Experiments demonstrate that VA‑TDM achieves state‑of‑the‑art performance for four‑step text‑to‑image and text‑to‑video generation, outperforming prior few‑step methods and approaching the quality of many‑step baselines.

By Yihong Luo, Tianyu Pang, Tianyang Hu