arXiv Machine Learning

Wasserstein Formulation of Reinforcement Learning. An Optimal Transport Perspective on Policy Optimization

The paper introduces a geometric framework for reinforcement learning that treats policies as mappings into the Wasserstein space of action probabilities. It establishes a Riemannian structure induced by stationary distributions, defines the tangent space of policies, and characterizes geodesics while addressing measurability concerns. The authors formulate a general RL optimization problem, construct a gradient flow via Otto's calculus, compute the gradient and Hessian of the energy, and demonstrate the approach with numerical examples for low‑dimensional problems and neural‑network‑parameterized policies for high‑dimensional settings.

arXiv Statistics ML
Oct 2

Convergence Analysis of the Wasserstein Proximal Algorithm beyond Geodesic Convexity

The paper presents a convergence analysis of the Wasserstein proximal algorithm without requiring geodesic convexity of the objective functional. By assuming a Wasserstein analog of the Euclidean Polyak–Łojasiewicz inequality, the authors prove that the algorithm achieves unbiased, linear convergence, improving on previous rates that required strong geodesic convexity. The analysis is extended to inexact proximal steps for geodesically semiconvex objectives, and numerical experiments show faster convergence than noisy gradient descent on mean‑field neural networks.

By Shuailong Zhu, Xiaohui Chen
arXiv Machine Learning
Sep 15

Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning

The review explores how control theory, optimal transport, probabilistic inference, non‑equilibrium thermodynamics, and machine learning are interconnected through the optimization of free‑energy‑like functionals under dynamical or statistical constraints. It presents a conceptual thread linking these five fields and illustrates the ideas with applications in reinforcement learning, variational inference, and generative modeling. The article is written for readers without prior familiarity, beginning with physics principles.

By Emmy Blumenthal, Nikolas Claussen, Benjamin Eysenbach, Catherine Ji, Gautam Reddy, Colin Scheibner, Benjamin Sorkin
arXiv Machine Learning
Sep 15

Stochastic Gradient Descent over P2

The paper develops a diffusion approximation for stochastic gradient descent (SGD) when the optimization target is a functional on the Wasserstein space ℝ2. By lifting the problem to a Hilbert space via Lions differentiability, the authors construct a Gaussian random-field approximation whose velocity field matches the mean and covariance of the original stochastic gradient. They prove that this Gaussian approximation achieves second‑order weak accuracy, providing a rigorous basis for replacing sample‑driven randomness with analytically tractable Gaussian fluctuations in stochastic optimization over probability measures.

By Maria Oprea, Qin Li, Yunan Yang
arXiv Machine Learning
Oct 5

Light Entropic Optimal Transport on Riemannian Manifolds

Light Entropic Optimal Transport on Riemannian Manifolds introduces ManifoldLightOT, a lightweight method for learning kernel‑induced entropic optimal transport couplings directly on common manifolds such as spheres, tori, SO(3), and SE(3). By constructing geometry‑specific Gibbs kernels and compatible potential parameterizations, the approach yields closed‑form normalization and directly sampleable conditional distributions. The formulation extends naturally to products of manifolds and optimizes potentials from samples using Monte Carlo estimates, demonstrating superior performance over existing manifold OT methods in synthetic and real‑world experiments.

By Xavier Aramayo-Carrasco, Petr Mokrov, Alexander Korotin
arXiv AI
Jun 24

The Geometry Behind Diffusion and Flow Matching: Gradient Flows and Geodesics in Wasserstein Space

arXiv:2606. 24157v1 Announce Type: new Abstract: The space $\mathcal{P}_2(\mathbb{R}^d$) of probability measures with finite second moment carries a natural geometry: the quadratic Wasserstein distance W_2 makes it a complete metric space and, following Otto, a (formal) Riemannian manifold whose geodesics are the optimal-transport interpolations.

By Yian Yao, Weiwei Zhang
arXiv AI
Jun 16

Optimal Transport for Machine Learners

arXiv:2505. 06589v2 Announce Type: replace-cross Abstract: Modern machine learning repeatedly manipulates probability measures: empirical datasets, generated samples, latent distributions, class-conditional laws, particle systems, weights of wide networks and attention patterns.

By Gabriel Peyr\'e