arXiv Machine Learning

Critically-Damped Higher-Order Langevin Dynamics for Generative Modeling

The paper introduces Critically‑Damped Higher‑Order Langevin Dynamics (C‑HOLD) for generative modeling, extending the concept of critical damping to arbitrary‑order Langevin dynamics. It builds on the recent Higher‑Order Langevin Dynamics (HOLD) method by providing an optimal hyperparameter set for the n‑dimensional case and closed‑form expressions for the mean and covariance of the forward process. Experiments on CIFAR‑10 and CelebA‑HQ 256×256 datasets demonstrate the approach’s effectiveness using the FID metric.

arXiv Machine Learning
Oct 16

Iterative Neural Variational Solver of Committor Function in Under-damped Langevin Dynamics

The paper introduces an iterative neural variational solver for the committor function in under‑damped Langevin dynamics. By splitting the infinitesimal generator into self‑adjoint and skew‑adjoint parts, each subproblem gains a variational form that can be efficiently solved with neural networks, resembling a generalized Concus‑Golub‑Widlund splitting. The method is validated on benchmark systems such as a double‑well potential, the Muller‑Brown potential, and alanine dipeptide, demonstrating practical accuracy and extensibility for studying rare transition events across different damping coefficients.

By Tiejun Li, Zhaoqi Liu, Xiang Zhou
arXiv Statistics ML
Sep 28

First-Order Stationarity of Reverse Diffusions

The paper establishes a first‑order theoretical framework for diffusion models, showing that SDE‑based reverse‑time flows of both overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates when the stationary potential of the forward process is strongly convex. It further incorporates discretization to provide averaged first‑order stationarity bounds—sampling analogues of averaged gradient‑norm guarantees in nonconvex optimization—for samplers of both diffusion models. These results highlight a unique advantage of SDE‑based reverse diffusion over ODE‑based approaches, offering local convexity‑free certificates that ensure score consistency rather than global mode weights.

By Zhifeng Chen, Chenyang Jiang, Yazhen Wang
arXiv Statistics ML
Sep 25

Global Convergence of Third-Order Langevin Dynamics for Non-Convex Optimization via Simulated Annealing

The paper establishes global convergence guarantees for third‑order Langevin dynamics applied to non‑convex optimization via simulated annealing with fixed friction and decreasing noise. It shows that, under dissipativity, regularity, and low‑temperature functional‑inequality assumptions, a logarithmic cooling schedule drives objective values to the global minimum with a barrier‑controlled kinetic rate, and that polynomially decreasing step sizes preserve this rate for both exact‑force‑integral and midpoint three‑stage discretizations. Numerical experiments on a double‑well problem and high‑dimensional neural‑network objectives demonstrate that third‑order Langevin schemes outperform overdamped Langevin dynamics and, in some cases, the one‑gradient UBU integrator in terms of terminal‑success point estimates and post‑quench test accuracy.

By Yingli Wang, Kelvin Shuangjian Zhang, Lingjiong Zhu
arXiv Machine Learning
Aug 27

Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling

The paper introduces Hessian-free high-resolution (HFHR) dynamics, an extension of underdamped Langevin dynamics that incorporates reversible position diffusion for sampling in machine learning. It provides an explicit quantitative contraction rate under a position Poincaré inequality, weighted Hessian and Laplacian bounds, and a compact Sobolev embedding, even when the potential is non‑convex. For the HFHR Monte Carlo algorithm, a path‑space Girsanov argument yields a non‑asymptotic convergence bound and an explicit iteration complexity in total variation distance, improving on previous HFHR results and demonstrating benefits of a positive diffusion parameter through numerical experiments.

By Wujun Lv, Xiaoyu Wang, Yingli Wang, Lingjiong Zhu
arXiv Machine Learning
Oct 8

Reflected Anchored Langevin Algorithms

The paper introduces Reflected Anchored Langevin Dynamics (RALD), a reflected diffusion that can converge to non‑differentiable target distributions on constrained domains. By employing a smooth anchored reference potential and scaling the drift and noise covariance with a state‑dependent factor, the authors derive an Euler‑Maruyama discretization with projection, yielding the Reflected Anchored Langevin Monte Carlo (RALMC) algorithm. Explicit convergence bounds and iteration complexity in the 2‑Wasserstein distance are proven, and numerical experiments confirm the theoretical predictions and empirical performance.

By Changwei Tu, Xiaoyu Wang, Yingli Wang, Xicheng Zhang, Lingjiong Zhu