arXiv Machine Learning By Jiahao Yu, Song Liu, Jos\'{e} Miguel Hern\'{a}ndez-Lobato, RuiKang OuYang

Controlling Dependence in Implicit Generative Models via Spread Mutual Information

Read the original on arXiv Machine Learning →

The paper introduces Spread Mutual Information (SMI), a new metric that integrates mutual information over noise levels using a spreading kernel. By applying Gaussian spreading, SMI ensures smooth, strictly positive conditional and marginal densities, enabling gradient estimation even for originally singular distributions. Experiments show that SMI consistently outperforms other MI-based methods for controlling dependence in implicit generative models and competes well with task‑specific approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Oct 28

Active Learning of Conditional Generative Models via the Transport Neural Tangent Kernel

The paper introduces the transport Neural Tangent Kernel (tNTK) to enable active learning for conditional generative models that produce distributional outputs. By decomposing the Wasserstein test risk into mean and shape components, the authors show that mean‑based active learning misses distributional shape, and propose tNTK as a tractable kernel that captures sensitivity of the predicted distribution to model parameters. Using tNTK as a Gaussian process covariance, they greedily select inputs that minimize posterior variance, and demonstrate that this distribution‑aware strategy outperforms mean‑based NTK baselines across several generative model types and datasets.

By Jayoung Ryu, Kyunghyun Cho, Romain Lopez
arXiv Machine Learning
Jul 20

Diffusion models recover accurate mixture weights despite score function insensitivity

arXiv:2607. 15485v1 Announce Type: new Abstract: Score-based generative models exhibit a puzzling behavior: they often appear to cover all modes of a target multimodal distribution and yet may fail to learn the correct relative mode amplitudes, which can be interpreted as mixture weights.

By Andrew Dennehy, Ramchandran Muthukumar, Rebecca Willett, Nisha Chandramoorthy
arXiv Machine Learning
Oct 16

Efficient Agnostic Learning of Univariate Densities Under Local Differential Privacy

The paper introduces a general framework for non‑interactive locally differentially private (LDP) learning of structured univariate distributions that can be approximated by piece‑wise polynomials. It achieves the first sample‑efficient LDP learners for several distribution classes, notably mixtures of log‑concave distributions such as Gaussian mixture models (GMMs), while providing agnostic guarantees and computational efficiency. The authors also establish a new lower bound for learning GMMs under LDP, surpassing previous pure‑DP packing bounds, and develop an efficient non‑interactive LDP algorithm for synthetic data generation over large domains, which is then combined with a deterministic adaptation of a non‑private piecewise polynomial learner.

By Mohammad Afzali, Hassan Ashtiani, Sivan Sabato
arXiv AI
Jul 29

Generative Distributionally Robust Optimization

arXiv:2607. 24983v1 Announce Type: cross Abstract: Generative models are increasingly adopted in distributionally robust optimization (DRO), but existing approaches trade off model compatibility and adversarial structure: methods that accept arbitrary samplers do not restrict worst-case laws to a generator family, while generator-parameterized adversaries rely on model-specific access such as likelihoods, scores, or training data.

By Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao Jin
arXiv Machine Learning
Oct 22

MCLR: Improving Conditional Modeling via Inter-Class Likelihood-Ratio Maximization and Unifying Classifier-Free Guidance with Alignment Objectives

arXiv:2603.22364v4 Announce Type: replace Abstract: Diffusion models achieve strong generative performance but often rely on classifier-free guidance (CFG), an inference-time intervention that modifi...

By Xiang Li, Yixuan Jia, Xiao Li, Jeffrey A. Fessler, Rongrong Wang, Qing Qu
arXiv AI
Jun 24

Catastrophic Compositional Generation: Why Vanilla Diffusion Models Fail to Extrapolate

arXiv:2606. 23920v1 Announce Type: cross Abstract: The task of compositional generation involves using a conditional generative model, trained only on a subset of the possible conditions, to produce samples from compositionally-defined target distributions such as a geometric combination of the source distributions.

By Duncan Soiffer, Chandler Squires, Yuan Guan, Jason Hartford, Pradeep Ravikumar