arXiv:2608. 16333v1 Announce Type: cross Abstract: On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories.
By Changhui Sun, Lanbo Liu, Hang Lei, Tong Ling, Jiahang Xie, Zhiyong Zheng, Yujia Wang, Hao Liu, Feng Xiao, Lu Liu, Yanlong Du, Zifeng Cheng, Ziwei Jiang, Qing Gu
On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories. This approach has achieved strong empirical gains and can often surpass conve...
The paper introduces Velocity Adversarial Distillation (VAD), a method that uses a learned critic to compare velocity fields of student and teacher diffusion models, turning disagreements into learning signals. VAD unifies several existing distillation techniques and leads to VA‑TDM, which applies trajectory distribution matching and transition‑aware online velocity learning to supervise partial denoising along the student’s sampling path. Experiments demonstrate that VA‑TDM achieves state‑of‑the‑art performance for four‑step text‑to‑image and text‑to‑video generation, outperforming prior few‑step methods and approaching the quality of many‑step baselines.
By Yihong Luo, Tianyu Pang, Tianyang Hu
arXiv:2609.36638v1 Announce Type: new
Abstract: Text-to-image users often provide concise and underspecified prompts, whereas generative models benefit from detailed textual conditions for reliable i...
By Mingfeng Lin, Chengfei Cai, Lin Xu, Chengqian Ma, Yuxiang Wei, Liang Han
arXiv:2606. 27377v1 Announce Type: cross Abstract: Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing.
By Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, Tat-Seng Chua
On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules.
arXiv:2607. 24522v1 Announce Type: new Abstract: While on-policy distillation (OPD) effectively addresses sparse rewards and exposure bias in large language model post-training, its extension to flow models remains underexplored.
By Kaiyang Ye, Yuan Ge, Junxiang Zhang, Bei Li, Ziming Zhu, Haishu Zhao, Xiaoqian Liu, Chenglong Wang, Jingbo Zhu, Zhengtao Yu, Tong Xiao
A deployable text-to-image diffusion model must hold several capabilities at once, such as instruction following, text rendering and human preference, and must sample in a few steps. Capabilities and...
arXiv:2608. 03316v1 Announce Type: new Abstract: On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid.
By Siming Fu, Zheming Fu, Ruizhe He, Hualiang Wang, Jie Huang, Xiaoxiao Ma, Mingchen Zhong, Weihu Huang, Xiaoxuan He, Haojun Xu
arXiv:2608. 09233v1 Announce Type: new Abstract: Flow-matching models are now a mainstream method to image generation, but its adaptation to diverse downstream scenarios typically relies on post-training, which may cause conflicts among task-specific optimization objectives.
By Mingfeng Lin, Chengfei Cai, Lin Xu, Yuxiang Wei, Liang Han
Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce Fl...
CA-OPD is a confidence‑aware on‑policy distillation framework that improves structured visual prediction by using teacher confidence to selectively correct unreliable student transitions and gradually transfer rollout control to the student. The method aligns supervision with intervention decisions, providing direct cross‑entropy loss for corrected tokens and full predictive distribution for retained tokens. In a multi‑teacher setting for GUI grounding and OCR, CA‑OPD significantly outperforms the Qwen3.5‑0.8B baseline, achieving large gains on benchmarks such as ScreenSpot‑Pro and OCRBench‑v2 English.
By Menghao Li, Linjie Mu, Yin Wang, Haotian Hu, Yannian Gu, Lujiayi Xue, Fanyi Wang