← Back to all news
arXiv Statistics ML By Yingli Wang, Kelvin Shuangjian Zhang, Lingjiong Zhu

Global Convergence of Third-Order Langevin Dynamics for Non-Convex Optimization via Simulated Annealing

Read the original on arXiv Statistics ML →

The paper establishes global convergence guarantees for third‑order Langevin dynamics applied to non‑convex optimization via simulated annealing with fixed friction and decreasing noise. It shows that, under dissipativity, regularity, and low‑temperature functional‑inequality assumptions, a logarithmic cooling schedule drives objective values to the global minimum with a barrier‑controlled kinetic rate, and that polynomially decreasing step sizes preserve this rate for both exact‑force‑integral and midpoint three‑stage discretizations. Numerical experiments on a double‑well problem and high‑dimensional neural‑network objectives demonstrate that third‑order Langevin schemes outperform overdamped Langevin dynamics and, in some cases, the one‑gradient UBU integrator in terms of terminal‑success point estimates and post‑quench test accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.

Related stories

arXiv Machine Learning
Aug 27

Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling

The paper introduces Hessian-free high-resolution (HFHR) dynamics, an extension of underdamped Langevin dynamics that incorporates reversible position diffusion for sampling in machine learning. It provides an explicit quantitative contraction rate under a position Poincaré inequality, weighted Hessian and Laplacian bounds, and a compact Sobolev embedding, even when the potential is non‑convex. For the HFHR Monte Carlo algorithm, a path‑space Girsanov argument yields a non‑asymptotic convergence bound and an explicit iteration complexity in total variation distance, improving on previous HFHR results and demonstrating benefits of a positive diffusion parameter through numerical experiments.

By Wujun Lv, Xiaoyu Wang, Yingli Wang, Lingjiong Zhu
ragdiffusion
More like this →
arXiv Machine Learning
Sep 23

Penalized Nonreversible Langevin for Constrained Sampling

The paper introduces penalized nonreversible Langevin algorithms for sampling from a target distribution constrained to a compact convex set. It combines a squared distance penalty with skew-symmetric perturbations that preserve the penalized Gibbs distribution, and provides nonasymptotic total variation and Wasserstein bounds under various smoothness and contraction assumptions. Numerical experiments demonstrate the methods on constrained Bayesian regression, classification, neural networks, and truncated sampling, highlighting acceleration in a stochastic quadratic model.

By Pervez Ali, Weihao Dong, Xiaoyu Wang
More like this →