arXiv Machine Learning

Controlling Dependence in Implicit Generative Models via Spread Mutual Information

The paper introduces Spread Mutual Information (SMI), a new metric that integrates mutual information over noise levels using a spreading kernel. By applying Gaussian spreading, SMI ensures smooth, strictly positive conditional and marginal densities, enabling gradient estimation even for originally singular distributions. Experiments show that SMI consistently outperforms other MI-based methods for controlling dependence in implicit generative models and competes well with task‑specific approaches.

arXiv Machine Learning
Oct 28

Active Learning of Conditional Generative Models via the Transport Neural Tangent Kernel

The paper introduces the transport Neural Tangent Kernel (tNTK) to enable active learning for conditional generative models that produce distributional outputs. By decomposing the Wasserstein test risk into mean and shape components, the authors show that mean‑based active learning misses distributional shape, and propose tNTK as a tractable kernel that captures sensitivity of the predicted distribution to model parameters. Using tNTK as a Gaussian process covariance, they greedily select inputs that minimize posterior variance, and demonstrate that this distribution‑aware strategy outperforms mean‑based NTK baselines across several generative model types and datasets.

By Jayoung Ryu, Kyunghyun Cho, Romain Lopez
arXiv Machine Learning
Jul 20

Diffusion models recover accurate mixture weights despite score function insensitivity

arXiv:2607. 15485v1 Announce Type: new Abstract: Score-based generative models exhibit a puzzling behavior: they often appear to cover all modes of a target multimodal distribution and yet may fail to learn the correct relative mode amplitudes, which can be interpreted as mixture weights.

By Andrew Dennehy, Ramchandran Muthukumar, Rebecca Willett, Nisha Chandramoorthy
arXiv Machine Learning
Oct 16

Efficient Agnostic Learning of Univariate Densities Under Local Differential Privacy

The paper introduces a general framework for non‑interactive locally differentially private (LDP) learning of structured univariate distributions that can be approximated by piece‑wise polynomials. It achieves the first sample‑efficient LDP learners for several distribution classes, notably mixtures of log‑concave distributions such as Gaussian mixture models (GMMs), while providing agnostic guarantees and computational efficiency. The authors also establish a new lower bound for learning GMMs under LDP, surpassing previous pure‑DP packing bounds, and develop an efficient non‑interactive LDP algorithm for synthetic data generation over large domains, which is then combined with a deterministic adaptation of a non‑private piecewise polynomial learner.

By Mohammad Afzali, Hassan Ashtiani, Sivan Sabato
arXiv AI
Jul 29

Generative Distributionally Robust Optimization

arXiv:2607. 24983v1 Announce Type: cross Abstract: Generative models are increasingly adopted in distributionally robust optimization (DRO), but existing approaches trade off model compatibility and adversarial structure: methods that accept arbitrary samplers do not restrict worst-case laws to a generator family, while generator-parameterized adversaries rely on model-specific access such as likelihoods, scores, or training data.

By Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao Jin
arXiv Machine Learning
Oct 22

MCLR: Improving Conditional Modeling via Inter-Class Likelihood-Ratio Maximization and Unifying Classifier-Free Guidance with Alignment Objectives

arXiv:2603.22364v4 Announce Type: replace Abstract: Diffusion models achieve strong generative performance but often rely on classifier-free guidance (CFG), an inference-time intervention that modifi...

By Xiang Li, Yixuan Jia, Xiao Li, Jeffrey A. Fessler, Rongrong Wang, Qing Qu
arXiv AI
Jun 24

Catastrophic Compositional Generation: Why Vanilla Diffusion Models Fail to Extrapolate

arXiv:2606. 23920v1 Announce Type: cross Abstract: The task of compositional generation involves using a conditional generative model, trained only on a subset of the possible conditions, to produce samples from compositionally-defined target distributions such as a geometric combination of the source distributions.

By Duncan Soiffer, Chandler Squires, Yuan Guan, Jason Hartford, Pradeep Ravikumar
arXiv Machine Learning
Jul 31

Error Analysis of Neural-Network-Based Engression

arXiv:2607. 27723v1 Announce Type: cross Abstract: Engression (Shen and Meinshausen, 2024) learns a conditional distribution by fitting a generative model $Y = f(X,\varepsilon)$ under the energy score, a strictly proper scoring rule.

By Juntong Chen, Zijian Guo, Xinwei Shen
arXiv Statistics ML
Oct 16

Optimal score function estimation via derivatives constraints

The paper develops a statistical framework showing that empirical score matching can achieve minimax optimal performance for diffusion generative models when data lie on a low‑dimensional manifold. It establishes convergence rates for empirical risk minimization over constrained Sobolev classes and demonstrates that the resulting score estimators yield minimax optimal Wasserstein‑1 rates for the generated distribution, up to logarithmic factors. The analysis reveals that the statistical rates depend on the intrinsic dimension of the data manifold rather than its ambient dimension, providing conditions under which score‑based generative modeling generalizes rather than overfits.

By Thomas Bonis, Thanh Mai Pham Ngoc, Viet Chi Tran