arXiv Statistics ML

Provably Reliable Classifier Guidance via Cross-Entropy Control

arXiv AI
Jun 10

MMD Guidance: Training-Free Distribution Adaptation for Diffusion Models via Maximum Mean Discrepancy Guidance

arXiv:2601. 08379v2 Announce Type: replace-cross Abstract: Pre-trained diffusion models have emerged as powerful generative priors for both unconditional and conditional sample generation, yet their outputs often deviate from the characteristics of user-specific target data.

By Matina Mahdizadeh Sani, Nima Jamali, Mohammad Jalali, Farzan Farnia
arXiv Machine Learning
Oct 12

Opening the Black Box of Classifier-Free Guidance via Information Bottleneck

The paper investigates when to activate classifier‑free guidance (CFG) in diffusion models by treating the diffusion trajectory as a time‑dependent information bottleneck. It shows that the usefulness of conditional information increases after a critical stage defined by a Fisher‑information balance, and proposes a training‑free dynamic CFG strategy that activates guidance based on a stochastic process of score statistics. Experiments across various architectures and tasks confirm that this adaptive rule improves effectiveness, robustness, and general applicability while offering a principled explanation for dynamic guidance.

By Jiayang Gao, Tianyi Zheng, Jiayang Zou, Fengxiang Yang, Luyao Fan, Lv Tang, Bo Li, Jia Wang
arXiv Machine Learning
Oct 22

MCLR: Improving Conditional Modeling via Inter-Class Likelihood-Ratio Maximization and Unifying Classifier-Free Guidance with Alignment Objectives

arXiv:2603.22364v4 Announce Type: replace Abstract: Diffusion models achieve strong generative performance but often rely on classifier-free guidance (CFG), an inference-time intervention that modifi...

By Xiang Li, Yixuan Jia, Xiao Li, Jeffrey A. Fessler, Rongrong Wang, Qing Qu
arXiv Machine Learning
2d ago

Quantifying Error Propagation and Model Collapse in Diffusion Models

The paper investigates how repeatedly training score-based diffusion models on synthetic data can cause their outputs to drift from the target distribution. It provides theoretical upper and lower bounds on the divergence accumulated over successive training rounds, distinguishing regimes based on score estimation error and the fraction of fresh data used. Empirical experiments on synthetic data and images support the theoretical findings.

By Nail B. Khelifa, Richard E. Turner, Ramji Venkataramanan
arXiv Machine Learning
Oct 9

Memorization and Malign Generalization in Conditional Diffusion Models with Random Features

The paper studies memorization and generalization in conditional diffusion models using a random-feature conditional score model in a high-dimensional setting. It derives asymptotic expressions for training and test losses, showing that in overparameterized regimes wider models better predict the condition-dependent mean but also exhibit reduced within-condition variance, a behavior termed "malign generalization." The analysis also reveals that more informative conditions cause training samples to be memorized at smaller widths, with experimental validation on U‑Net architectures.

By Gwangho Kim, Sungyoon Lee
Hugging Face Trending Papers
Sep 3

Conditioning Degenerate Diffusion Models

The paper addresses the challenge of guiding conditioned generative models that are diffusion processes with singular diffusion coefficients, where traditional conditional densities may be nonexistent or non‑smooth. It proposes using causal optimal transport to construct approximate loss functions that identify a minimum‑entropy control for guidance, relying on the predictable representation property of conditioned diffusion processes and well‑posed martingale problems à la Üstünel.