arXiv AI

Reflected Proximal Dynamics for Accelerated High-Accuracy Log-Concave Sampling

arXiv Machine Learning
Oct 9

Accelerating Non-Smooth and Heavy-Tailed Sampling

The paper introduces non‑reversible anchored Langevin dynamics (NALD) and its reflected variant (NRALD) for sampling target distributions that may be non‑smooth and heavy‑tailed, both in Euclidean space and on constrained domains. By adding a circulation drift through a divergence‑free skew‑symmetric matrix field and a stream potential, the methods preserve the target distribution without needing density derivatives, admit a random‑time‑change representation, and can be applied with normal reflection on bounded domains. The authors demonstrate that breaking reversibility leads to faster convergence to the target distribution, as shown by finite‑time non‑asymptotic analysis, large‑deviations results, and asymptotic variance reduction, with numerical experiments confirming the algorithms’ efficiency.

By Pervez Ali, Xiaoyu Wang, Yingli Wang, Lingjiong Zhu
arXiv Machine Learning
Sep 23

Penalized Nonreversible Langevin for Constrained Sampling

The paper introduces penalized nonreversible Langevin algorithms for sampling from a target distribution constrained to a compact convex set. It combines a squared distance penalty with skew-symmetric perturbations that preserve the penalized Gibbs distribution, and provides nonasymptotic total variation and Wasserstein bounds under various smoothness and contraction assumptions. Numerical experiments demonstrate the methods on constrained Bayesian regression, classification, neural networks, and truncated sampling, highlighting acceleration in a stochastic quadratic model.

By Pervez Ali, Weihao Dong, Xiaoyu Wang
arXiv Machine Learning
Sep 10

Smoothed Picard Hamiltonian Monte Carlo

arXiv:2609.06906v1 Announce Type: cross Abstract: We develop a new low-accuracy sampler, called \emph{smoothed Picard Hamiltonian Monte Carlo}, which combines Gaussian smoothing, Picard iteration, an...

By Fan Chen, Sinho Chewi, Jianfeng Lu, Matthew S Zhang
arXiv Machine Learning
Oct 5

ENCORE: Exact Non-equilibrium COntrol with Replica Exchange for Diffusion Generation

ENCORE is a new exact parallel control method for inference‑time steering of pretrained generative models toward tilted target distributions. It uses replica exchange with each replica storing its trajectory, avoiding simulation of the intractable time reversal and ensuring target invariance. Experiments on synthetic, biomolecular, and image data show that ENCORE achieves competitive accuracy and diversity, remains robust to sampler perturbations, and works with distilled samplers where prior replica‑exchange corrections fail.

By Jiahao Yu, Saifuddin Syed, Jos\'e Miguel Hern\'andez-Lobato, Jiajun He
arXiv Machine Learning
Oct 16

Iterative Neural Variational Solver of Committor Function in Under-damped Langevin Dynamics

The paper introduces an iterative neural variational solver for the committor function in under‑damped Langevin dynamics. By splitting the infinitesimal generator into self‑adjoint and skew‑adjoint parts, each subproblem gains a variational form that can be efficiently solved with neural networks, resembling a generalized Concus‑Golub‑Widlund splitting. The method is validated on benchmark systems such as a double‑well potential, the Muller‑Brown potential, and alanine dipeptide, demonstrating practical accuracy and extensibility for studying rare transition events across different damping coefficients.

By Tiejun Li, Zhaoqi Liu, Xiang Zhou
arXiv Machine Learning
Oct 12

Robust Langevin Sampling under State-Dependent Heavy-Tailed Noise with Applications to LLM Fine-Tuning

arXiv:2610. 12865v1 Announce Type: new Abstract: Existing convergence guarantees for non-log-concave sampling with stochastic gradient Langevin Monte Carlo (SG-LMC) typically assume globally Lipschitz gradients and uniformly bounded noise variance.

By M. Berk Sahin, Ege C. Kaya, Arda Fazla, Behzad Sharif, Abolfazl Hashemi
arXiv Statistics ML
Aug 25

Provably adaptive sampling with uniform and remasking discrete diffusion models

The paper proves that for discrete diffusion models using uniform or remasking forward processes, an adaptive sampler based on a leave‑one‑out denoiser can achieve sampling error proportional to the score‑estimation error plus a small tolerance. The required number of discretization steps scales with the dual total correlation of the target distribution, not directly with the ambient dimension. This result shows that sampling complexity is governed by the intrinsic dependence structure of the distribution, and the authors provide an information‑theoretic analysis linking discretization error to mutual information between coordinates.

By Daniil Dmitriev, Zhihan Huang, Yuting Wei