← Back to all news
arXiv Machine Learning By Kevin Kurian Thomas Vaidyan, Ahmet Alacaoglu, Michael P. Friedlander

Last-Iterate Convergence of Stochastic Gradient Methods for Unbounded Problems

Read the original on arXiv Machine Learning →

The paper investigates last‑iterate convergence for proximal and projected stochastic gradient methods in settings without bounded domain or bounded variance assumptions, using black‑box variational inference as an example. It establishes a near‑optimal anytime rate of ×(1/√T) for the last iterate under two relaxed assumptions, and shows that when the horizon T is known, the logarithmic factor can be removed to achieve the optimal ×(1/√T) rate, matching results that previously required bounded variance or domain constraints.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jul 2

Towards Weaker Variance Assumptions for Stochastic Optimization

arXiv:2504. 09951v2 Announce Type: replace-cross Abstract: We revisit a classical assumption for analyzing stochastic gradient algorithms where the squared norm of the stochastic subgradient (or the variance for smooth problems) is allowed to grow as fast as the squared norm of the optimization variable.

By Ahmet Alacaoglu, Yura Malitsky, Stephen J. Wright
More like this →
arXiv Machine Learning
Oct 16

On the Last-Iterate Convergence of Clipped Gradient Methods

The paper establishes the first optimal last‑iterate convergence rates, in high probability, for Clipped‑SGD when the gradient noise is heavy‑tailed. It addresses the gap left by earlier results that required strong noise assumptions, such as exponentially decaying tails. By proving these rates, the authors provide theoretical support for the practical effectiveness of gradient clipping in stochastic convex optimization.

By Zijian Liu
More like this →