← Back to all news
arXiv Machine Learning By Jaesung R. Park, Uijeong Jang, Yifan Chen

Acceleration and its Limits in Gaussian Variational Inference

Read the original on arXiv Machine Learning →

The paper investigates whether momentum-based acceleration can improve Gaussian variational inference (GVI) for strongly log-concave, log-smooth targets. With exact Gaussian expectations, GVI achieves the classic accelerated √κ convergence rate, but when only pointwise derivatives are available, the authors construct targets that force any randomized algorithm to use κ^{1-o(1)} queries, showing curvature alone does not guarantee acceleration. They identify additional structure—specifically a Hessian-Lipschitz condition with small Hessian variation—under which variance-reduced stochastic methods can achieve accelerated guarantees, and demonstrate empirical speedups on real datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 12

A lower bound for stepsize-based acceleration of gradient descent

arXiv:2608. 10418v1 Announce Type: cross Abstract: Recent work has shown that, for smooth convex optimization, plain gradient descent can be accelerated from its textbook convergence rate of $O(T^{-1})$ (where $T$ denotes the number of iterations) to $O\big(T^{-\log_2(1+\sqrt{2})}\big)$ using carefully designed stepsize schedules alone, without resorting to momentum or other algorithmic modifications.

By Jianhao Ma, Yuxin Chen
llmsbenchmarks
More like this →
arXiv Machine Learning
Sep 23

Penalized Nonreversible Langevin for Constrained Sampling

The paper introduces penalized nonreversible Langevin algorithms for sampling from a target distribution constrained to a compact convex set. It combines a squared distance penalty with skew-symmetric perturbations that preserve the penalized Gibbs distribution, and provides nonasymptotic total variation and Wasserstein bounds under various smoothness and contraction assumptions. Numerical experiments demonstrate the methods on constrained Bayesian regression, classification, neural networks, and truncated sampling, highlighting acceleration in a stochastic quadratic model.

By Pervez Ali, Weihao Dong, Xiaoyu Wang
More like this →
arXiv AI
Jun 16

Variance Reduction for Non-Log-Concave Sampling with Applications to Inverse Problems

arXiv:2606. 16257v1 Announce Type: cross Abstract: Sampling from high-dimensional, non-log-concave distributions with unnormalized densities is a fundamental challenge in machine learning, particularly when the exact gradient of the potential is unavailable and must be approximated via stochastic gradients that exhibit high variance under a fixed budget of gradient computations per iteration.

By M. Berk Sahin, Ahmet Ege Tanriverdi, Behzad Sharif, Abolfazl Hashemi
More like this →