← Back to all news
arXiv Machine Learning By Mathias Dus (IRMA)

Wasserstein Formulation of Reinforcement Learning. An Optimal Transport Perspective on Policy Optimization

Read the original on arXiv Machine Learning →

The paper introduces a geometric framework for reinforcement learning that treats policies as mappings into the Wasserstein space of action probabilities. It establishes a Riemannian structure induced by stationary distributions, defines the tangent space of policies, and characterizes geodesics while addressing measurability concerns. The authors formulate a general RL optimization problem, construct a gradient flow via Otto's calculus, compute the gradient and Hessian of the energy, and demonstrate the approach with numerical examples for low‑dimensional problems and neural‑network‑parameterized policies for high‑dimensional settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Statistics ML
Oct 2

Convergence Analysis of the Wasserstein Proximal Algorithm beyond Geodesic Convexity

The paper presents a convergence analysis of the Wasserstein proximal algorithm without requiring geodesic convexity of the objective functional. By assuming a Wasserstein analog of the Euclidean Polyak–Łojasiewicz inequality, the authors prove that the algorithm achieves unbiased, linear convergence, improving on previous rates that required strong geodesic convexity. The analysis is extended to inexact proximal steps for geodesically semiconvex objectives, and numerical experiments show faster convergence than noisy gradient descent on mean‑field neural networks.

By Shuailong Zhu, Xiaohui Chen
More like this →
arXiv Machine Learning
Sep 15

Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning

The review explores how control theory, optimal transport, probabilistic inference, non‑equilibrium thermodynamics, and machine learning are interconnected through the optimization of free‑energy‑like functionals under dynamical or statistical constraints. It presents a conceptual thread linking these five fields and illustrates the ideas with applications in reinforcement learning, variational inference, and generative modeling. The article is written for readers without prior familiarity, beginning with physics principles.

By Emmy Blumenthal, Nikolas Claussen, Benjamin Eysenbach, Catherine Ji, Gautam Reddy, Colin Scheibner, Benjamin Sorkin
reinforcement-learning
More like this →