arXiv:2608. 06283v1 Announce Type: new Abstract: We study the problem of sampling from target distributions whose potentials are simultaneously non-smooth, subject to superlinear gradient growth, and non-convex.
We study the problem of sampling from target distributions whose potentials are simultaneously non-smooth, subject to superlinear gradient growth, and non-convex. We introduce the Subgradient Tamed Unadjusted Langevin Algorithm (SG-TULA), a discretisation of the Langevin diffusion that operates directly on subgradients, without relying on computationally demanding smoothing procedures.
arXiv:2606. 15835v1 Announce Type: cross Abstract: Diffusion models have achieved impressive empirical success in generative tasks, and their convergence theory is now relatively well understood.
The paper presents a non‑asymptotic analysis of Markov chain Monte Carlo (MCMC) algorithms that learn and apply a preconditioner based on either the target covariance or the expected Hessian of the target potential. It compares the finite‑time computational costs of these preconditioned schemes with unpreconditioned counterparts, providing guarantees for algorithms such as the Unadjusted Langevin Algorithm (ULA) and the proximal sampler. The analysis relies on a contraction assumption in the Wasserstein‑2 distance to formalize approximate independence and bridge modern MCMC theory with classical effective sample size heuristics.
The paper introduces penalized nonreversible Langevin algorithms for sampling from a target distribution constrained to a compact convex set. It combines a squared distance penalty with skew-symmetric perturbations that preserve the penalized Gibbs distribution, and provides nonasymptotic total variation and Wasserstein bounds under various smoothness and contraction assumptions. Numerical experiments demonstrate the methods on constrained Bayesian regression, classification, neural networks, and truncated sampling, highlighting acceleration in a stochastic quadratic model.
arXiv:2610. 14567v2 Announce Type: replace-cross Abstract: This paper develops strong Gaussian approximations (KMT type) for constant learning rate stochastic gradient descent (SGD).
The paper introduces Hessian-free high-resolution (HFHR) dynamics, an extension of underdamped Langevin dynamics that incorporates reversible position diffusion for sampling in machine learning. It provides an explicit quantitative contraction rate under a position Poincaré inequality, weighted Hessian and Laplacian bounds, and a compact Sobolev embedding, even when the potential is non‑convex. For the HFHR Monte Carlo algorithm, a path‑space Girsanov argument yields a non‑asymptotic convergence bound and an explicit iteration complexity in total variation distance, improving on previous HFHR results and demonstrating benefits of a positive diffusion parameter through numerical experiments.
arXiv:2606. 07496v1 Announce Type: new Abstract: Decentralized stochastic optimization is a fundamental paradigm for large-scale learning over networks, where agents communicate only with their neighbors and no central coordinator is required.
The paper studies how to allocate a fixed computational budget across the denoising steps of diffusion models to improve sample quality at deployment. It shows that the expected benefit of evaluating multiple candidates at a step can be decomposed into a step‑specific sensitivity and a universal sample‑size factor, and that the optimal allocation follows a water‑filling structure. Experiments demonstrate that this allocation achieves the same quality as a uniform strategy while reducing function evaluations by 20–50%.
The paper introduces non‑reversible anchored Langevin dynamics (NALD) and its reflected variant (NRALD) for sampling target distributions that may be non‑smooth and heavy‑tailed, both in Euclidean space and on constrained domains. By adding a circulation drift through a divergence‑free skew‑symmetric matrix field and a stream potential, the methods preserve the target distribution without needing density derivatives, admit a random‑time‑change representation, and can be applied with normal reflection on bounded domains. The authors demonstrate that breaking reversibility leads to faster convergence to the target distribution, as shown by finite‑time non‑asymptotic analysis, large‑deviations results, and asymptotic variance reduction, with numerical experiments confirming the algorithms’ efficiency.
arXiv:2407.02765v4 Announce Type: replace-cross Abstract: We study the distributed optimization problem over a graphon with a continuum of nodes, which is regarded as the limit of the distributed net...
arXiv:2609.35947v1 Announce Type: new Abstract: Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the...