arXiv AI

The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer

OpenAI Blog
Jun 16, 2016

Generative models

This post describes four projects that share a common theme of enhancing or using generative models, a branch of unsupervised learning techniques in machine learning. In addition to describing our work, this post will tell you a bit more about generative models: what they are, why they are important, and where they might be going.

arXiv Machine Learning
Aug 27

Generative Modeling by Minimizing the Wasserstein-2 Loss

This paper introduces a generative model that minimizes the second‑order Wasserstein loss (W₂) by solving a distribution‑dependent ordinary differential equation (ODE) whose dynamics involve the Kantorovich potential of the true data distribution and its current estimate. The authors prove that the time‑marginal laws of this ODE form a gradient flow for the W₂ loss, converging exponentially to the true data distribution, and propose an Euler scheme that recovers this gradient flow in the limit. An algorithm based on this scheme, combined with persistent training, is shown in experiments to outperform Wasserstein GANs in both low‑ and high‑dimensional settings when the level of persistent training is appropriately increased.

By Yu-Jui Huang, Zachariah Malik
arXiv Machine Learning
Oct 9

Barron Optimal Transport I: Generative Modeling

The paper introduces a new optimal transport framework that uses the Barron energy—an intrinsic neural network complexity measure—instead of the traditional kinetic energy. By modeling transport as deep residual networks that map a reference distribution to a target distribution, the authors define a new metric on probability measures and analyze its properties for generative modeling. They demonstrate that diffusion generative models are suboptimal in this Barron geometry and explore how adaptivity can improve alternative generative approaches.

By Evan Dogariu, Joan Bruna