Self-OPD introduces a teacher‑free on‑policy distillation framework for flow matching models, using the student’s own exploration to generate step‑wise supervision. At each timestep the deterministic next‑state prediction is branched into multiple stochastic SDE candidates, rolled out, and compared against a deterministic baseline to compute normalized advantages. The velocity field is then optimized with a pull‑push objective that attracts high‑advantage branches and repels low‑advantage ones, while multi‑objective alignment is achieved by fusing normalized scores at the reward level.
By Shiyi Zhang, Mushui Liu, Yunze Tong, Wanggui He, Siyu Zou, Jinlong Liu, Yunlong Yu, Jian Song, Hao Jiang, Pipei Huang, Bo Zheng
arXiv:2610.02188v1 Announce Type: cross
Abstract: Distribution Matching Distillation (DMD) trains a few-step student from the difference between separately estimated target and student scores, so it...
By Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian, Yizhi Wang, Angtian Wang, Yiding Yang, Bo Liu, Xin Li, Wenping Wang, Chongyang Ma
arXiv:2608. 11937v1 Announce Type: new Abstract: Foundation models for time-dependent partial differential equations (PDEs) are trained on large and diverse collections of physical systems and can generalize effectively to new downstream tasks.
By Daniel Musekamp, Boshra Ariguib, Andrei Manolache, Mathias Niepert
Consistency distillation has significantly accelerated the inference of diffusion models. In this work, we reveal an intriguing asymmetry: while Logit-Normal sampling priors are highly efficacious for standard iterative generation, consistency distillation exhibits a distinctly different difficulty profile (e.
arXiv:2604. 03873v4 Announce Type: replace Abstract: Black-box knowledge distillation for large language models presents a strict trade-off.
By Xiwen Chen, Jingjing Wang, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Yueyue Deng, Hejian Sang, Zhipeng Wang, Alborz Geramifard, Feng Luo
DyMD introduces a Distribution Matching Distillation framework that adapts teacher supervision and critic fitting to preserve interaction dynamics in few-step video generation. By employing temporal affinity–conditioned re‑noise sampling and dynamics‑guided fake‑score tracking, DyMD balances motion recovery with visual quality. The method distills a 14B teacher into a 1.3B student that achieves significant gains on embodied‑video benchmarks and downstream action planning tasks.
By Haojun Xu, Jie Huang, Xin Lu, Mingchen Zhong, Zihao Fan, Linjiang Huang, Si Liu