arXiv Machine Learning
The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
5d ago
The Cost of an Unknown Horizon for the Last Iterate of Clipped SGD under Heavy-Tailed Noise
arXiv:2610. 33381v2 Announce Type: new Abstract: Training is often stopped at a data-dependent time, so guarantees for the last iterate must hold at all times simultaneously.
arXiv Machine Learning
Oct 5
Gradient Descent's Last Iterate is Often (slightly) Suboptimal
arXiv:2604.13870v2 Announce Type: replace-cross Abstract: We consider the well-studied setting of minimizing a convex Lipschitz function using either gradient descent (GD) or its stochastic variant (...
arXiv Machine Learning
Sep 14
Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness
arXiv:2609. 12785v1 Announce Type: new Abstract: Classical convergence guarantees for stochastic gradient methods typically assume Lipschitz-smooth objectives and finite-variance gradient noise, both frequently violated in practice.
arXiv Machine Learning
Sep 10
Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration
arXiv:2609.09152v1 Announce Type: cross Abstract: We study how far gradient descent (GD) can be accelerated by predetermined nonnegative stepsizes in smooth convex optimization. Writing $p_{\mathrm{s...
arXiv Machine Learning
Sep 30
High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness
arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
arXiv Machine Learning
Jun 24
New Bounds for the Last Iterate of the Stochastic subGradient Method
arXiv:2606. 24879v1 Announce Type: cross Abstract: We study the last iterate of the stochastic subgradient method for one-dimensional convex Lipschitz objectives.