Hugging Face Trending Papers

Disagreement-Aware Distribution Matching Distillation for Few-Step Diffusion Models

Read the original on Hugging Face Trending Papers →

Disagreement‑Aware Distribution Matching Distillation (DA‑DMD) builds on Distribution Matching Distillation (DMD) for few‑step diffusion models by leveraging the teacher‑fake disagreement signal already computed during training. The method activates only in the high‑noise regime, modulating update strength based on disagreement magnitude, and introduces no extra models or inference changes. Experiments across diffusion backbones show consistent improvements in few‑step generation, with DA‑DMD boosting HPSv2 on SD3.5‑L from 28.23 to 30.93 and achieving a 64.4% average win rate over DMD2.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Oct 2

DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

DMAD (Distribution Matching as Adversarial Distillation) reinterprets distribution matching as a classification task, enabling a few‑step student model to learn log‑density ratios directly via two discriminator heads on a shared backbone. This eliminates the need for an auxiliary diffusion model, reducing memory and computation overhead. Experiments show DMAD achieves state‑of‑the‑art Fréchet Inception Distance scores on ImageNet‑64x64, COCO‑10K, and VBench, and outperforms competing few‑step methods in joint audio‑video generation on MiniMax‑H3‑33B.

By Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian, Yizhi Wang, Angtian Wang, Yiding Yang, Bo Liu, Xin Li, Wenping Wang, Chongyang Ma
arXiv Machine Learning
Oct 8

Consistent Distribution Matching for Data-Free Diffusion Distillation

Consistent Distribution Matching is a data‑free distillation technique that accelerates diffusion and flow models by unifying sample generation and score estimation in a single student network. The method requires only a frozen teacher and a trainable student, optimizing a single objective that provably leads to Wasserstein convergence of the student’s flow‑map to the teacher’s marginals. On ImageNet 256×256, it achieves an FID of 2.04 with one function evaluation and 1.37 with four evaluations after just 40 training epochs, outperforming existing data‑free baselines.

By Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao
arXiv AI
Jul 3

DemoPSD: Disagreement-Modulated Policy Self-Distillation

arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.

By Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, Linqi Song
arXiv Machine Learning
Oct 2

Visualizing Distribution Coverage in Generative Diffusion Models

The paper introduces pass@k, a metric that evaluates how well generative diffusion models cover their output distribution by measuring the probability that at least one of k independent samples meets a quality criterion. Using this metric, the authors show that while classifier‑free guidance improves single‑draw quality, its advantage diminishes or reverses as k increases, revealing a trade‑off between quality and distribution coverage. They further demonstrate that the choice of training objective in diffusion distillation—whether distribution‑matching or consistency/trajectory‑based—determines whether a few‑step model retains its teacher’s coverage or sacrifices it for higher single‑draw performance, a phenomenon that also appears in few‑step causal video generation.

By Yifei Wang, Xiaoyu Wu, Tsu-Jui Fu, Chen Chen, Liang-Chieh Chen, Zhe Gan, Chen Wei