arXiv Machine Learning

A Likely Geometry of Generative Models

arXiv AI
Oct 12

ReSFlow: Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation

ReSFlow is a generative model that learns geodesic trajectories on statistical manifolds to enable one‑step categorical generation. By rectifying continuous‑time flows to follow intrinsic Riemannian geometry, it reduces transport cost and eliminates the need for multi‑step numerical integration. Experiments across image, text, and biological datasets show that ReSFlow achieves superior one‑step generation quality compared to prior methods.

By Xiaohuan Jia, Renzhe Xu, Xiao Wang, Jiayun Wu, Shaohua Fan
arXiv Machine Learning
Jul 14

Riemannian Denoising Diffusion Probabilistic Models

arXiv:2505. 04338v3 Announce Type: replace Abstract: We propose Riemannian Denoising Diffusion Probabilistic Models (RDDPMs) for learning distributions on submanifolds of Euclidean space that are level sets of functions, including most of the manifolds relevant to applications.

By Zichen Liu, Wei Zhang, Christof Sch\"utte, Tiejun Li
arXiv AI
Jun 9

Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering

arXiv:2605. 24942v2 Announce Type: replace-cross Abstract: Steering a language model - intervening on its internal activations to change downstream behaviour - has recently expanded beyond linear interpolation to nonlinear methods such as angular and kernelized steering, which define intervention transformations without learning an explicit geometry over paths in activation space.

By Narmeen Oozeer, Shivam Raval, Philip Quirke, Manikandan Ravikiran, Jeff Phillips, Shriyash Upadhyay, Amirali Abdullah
arXiv Machine Learning
5d ago

Learning Normal Coordinates on Riemannian Manifolds with Flow Charts

The paper introduces flow charts, a conditional invertible neural network designed to approximate normal coordinates on Riemannian manifolds, enabling fast Euclidean approximations of exponential and logarithmic maps. It presents a novel nonnegative loss function that is zero exactly when the chart is normal, and demonstrates that flow charts match or surpass specialized neural geodesic interpolators across sixteen synthetic and data‑derived geometries. When pretrained, flow charts achieve a median 60–70× speedup over numerical baselines while maintaining comparable logarithmic map estimation error and improving numerical geodesic solver accuracy.

By Edmond Cunningham
arXiv Machine Learning
Oct 2

RW-Flow: One-Step Generation on Compact Manifolds via Wasserstein Gradient Flows

RW-Flow presents a new one‑step generative framework for data on compact manifolds, leveraging Wasserstein gradient flows. The authors derive a necessary and sufficient identifiability condition for velocity fields on compact, connected Riemannian manifolds, showing that a symmetric, Lipschitz‑continuous cost function yields identifiability iff its Gibbs kernel is nondegenerate. Experiments on geospatial events, protein and RNA torsion angles, and discretized manifolds demonstrate that RW‑Flow surpasses existing one‑step methods across most benchmark settings.

By Ualibyek Nurgulan, Seungwoo Yoo, Prin Phunyaphibarn, Minhyuk Sung
arXiv Machine Learning
Oct 2

Manifold-Aware Perturbations for Constrained Generative Modeling

The paper introduces a new method for perturbing data distributions in a way that respects equality constraints, allowing generative models to better handle constrained data. By adjusting the distribution while preserving the manifold geometry, the approach ensures support matches the ambient space dimension. Experiments with diffusion models and normalizing flows demonstrate improved data recovery and stable sampling across several tasks.

By Katherine Keegan, Lars Ruthotto