arXiv Machine Learning
The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
3d ago
Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness
arXiv:2609. 12785v1 Announce Type: new Abstract: Classical convergence guarantees for stochastic gradient methods typically assume Lipschitz-smooth objectives and finite-variance gradient noise, both frequently violated in practice.
arXiv Machine Learning
Sep 10
Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration
arXiv:2609.09152v1 Announce Type: cross Abstract: We study how far gradient descent (GD) can be accelerated by predetermined nonnegative stepsizes in smooth convex optimization. Writing $p_{\mathrm{s...
arXiv Machine Learning
Jun 24
New Bounds for the Last Iterate of the Stochastic subGradient Method
arXiv:2606. 24879v1 Announce Type: cross Abstract: We study the last iterate of the stochastic subgradient method for one-dimensional convex Lipschitz objectives.
arXiv Machine Learning
6d ago
Silver Rate Is (Almost) Optimal for Gradient Descent
arXiv:2609. 09152v2 Announce Type: replace-cross Abstract: We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization.
arXiv Machine Learning
Jul 2
Towards Weaker Variance Assumptions for Stochastic Optimization
arXiv:2504. 09951v2 Announce Type: replace-cross Abstract: We revisit a classical assumption for analyzing stochastic gradient algorithms where the squared norm of the stochastic subgradient (or the variance for smooth problems) is allowed to grow as fast as the squared norm of the optimization variable.
arXiv Machine Learning
Aug 27
Beyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules
arXiv:2608. 25551v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory.