Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce Fl...
The paper introduces Velocity Adversarial Distillation (VAD), a method that uses a learned critic to compare velocity fields of student and teacher diffusion models, turning disagreements into learning signals. VAD unifies several existing distillation techniques and leads to VA‑TDM, which applies trajectory distribution matching and transition‑aware online velocity learning to supervise partial denoising along the student’s sampling path. Experiments demonstrate that VA‑TDM achieves state‑of‑the‑art performance for four‑step text‑to‑image and text‑to‑video generation, outperforming prior few‑step methods and approaching the quality of many‑step baselines.
By Yihong Luo, Tianyu Pang, Tianyang Hu
The paper introduces Trajectory-Regularized Distribution Distillation (TRD), a method that merges distribution-level and trajectory-level supervision for one-step generative models. TRD uses a momentum-mixture policy derived from a single student forward pass, eliminating the need for an extra network to estimate the student’s score. Experiments on SANA-1.6B and Stable Diffusion 3.5 demonstrate that TRD surpasses existing consistency and distribution matching baselines in generation quality and training efficiency, though it slightly reduces sample diversity.
By An Zhao, Yixiang Zhou, Zejian Li, Yi Guo, Pei Chen, Lingyun Sun
arXiv:2607. 24522v1 Announce Type: new Abstract: While on-policy distillation (OPD) effectively addresses sparse rewards and exposure bias in large language model post-training, its extension to flow models remains underexplored.
By Kaiyang Ye, Yuan Ge, Junxiang Zhang, Bei Li, Ziming Zhu, Haishu Zhao, Xiaoqian Liu, Chenglong Wang, Jingbo Zhu, Zhengtao Yu, Tong Xiao
arXiv:2608. 09233v1 Announce Type: new Abstract: Flow-matching models are now a mainstream method to image generation, but its adaptation to diverse downstream scenarios typically relies on post-training, which may cause conflicts among task-specific optimization objectives.
By Mingfeng Lin, Chengfei Cai, Lin Xu, Yuxiang Wei, Liang Han
Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamic...
arXiv:2606. 27377v1 Announce Type: cross Abstract: Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing.
By Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, Tat-Seng Chua
arXiv:2607. 00535v1 Announce Type: cross Abstract: Few-step flow-map generators, such as consistency models and MeanFlow, accelerate sampling by directly learning long-range transport maps between noise and data.
By Zhiqi Li, Wen Zhang, Bo Zhu
arXiv:2609.36638v1 Announce Type: new
Abstract: Text-to-image users often provide concise and underspecified prompts, whereas generative models benefit from detailed textual conditions for reliable i...
By Mingfeng Lin, Chengfei Cai, Lin Xu, Chengqian Ma, Yuxiang Wei, Liang Han
The paper investigates how the critic in Distribution Matching Distillation (DMD) should be designed to provide a reliable training signal. Experiments on ImageNet with seven SiT model sizes show that keeping a small critic while increasing teacher size can degrade student quality and cause training collapse; scaling the critic together with the teacher (joint scaling) prevents this collapse and lets students benefit from larger teachers. Additionally, the authors propose a step‑aware critic (STAC) that conditions on the student’s generation step, and demonstrate that joint scaling combined with STAC consistently improves few‑step distillation for class‑conditional ImageNet generation, text‑to‑image generation with SDXL and FLUX.2, and text‑to‑video generation with Wan2.1.
By Zongze Wu (Adobe), Xianbang Wang (MIT), Zhengyang Geng (Adobe), Mingdeng Cao (Adobe), Zhoutong Zhang (Adobe), Richard Zhang (Adobe), Eli Shechtman (Adobe), Ruiming Cao (Adobe)
The paper introduces VDOT++, a unified distillation framework for few-step video generation across text-to-video, image-to-video, and condition-based tasks. It extends the previous VDOT method by incorporating an asymmetric unbalanced optimal transport distillation that allows student tokens to carry less mass, and an α1 ground cost that uses a weighted median to preserve modes. The approach combines distribution matching with adversarial refinement and cross-scale distillation, achieving competitive performance on multiple video benchmarks with only four sampling steps.
By Yutong Wang, Xingtong Ge, Enhuai Liu, Yunke Wang, Tianfan Xue, Yu Qiao, Yaohui Wang, Xinyuan Chen, Chang Xu
The paper introduces HB‑SJD, a batched Speculative Jacobi Decoding backend that accelerates visual on‑policy distillation (OPD) by allowing images to advance independently and processing multiple tokens in parallel without a draft model. HB‑SJD switches between Full and Compact execution as images finish, reducing rollout and overall training time while maintaining generation quality. Experiments with LlamaGen demonstrate significant speedups without altering the teacher, distillation objective, or optimization procedure.
By Bingqi Shan, Zhehao Yu, Kenhong Lin, Baoquan Zhang