arXiv Machine Learning
Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
6d ago
Silver Rate Is (Almost) Optimal for Gradient Descent
arXiv:2609. 09152v2 Announce Type: replace-cross Abstract: We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization.
arXiv Machine Learning
Sep 3
Improved Gradient Descent Lower Bounds Beyond Nesterov
The paper investigates the limits of accelerating gradient descent (GD) using predetermined step sizes in smooth convex optimization. It establishes new lower bounds: an ≥·n−1.6342 non‑anytime bound and an ≥·n−1.2408 anytime bound, surpassing previous results. These findings also demonstrate a strict separation between convergence exponents achievable in non‑anytime versus anytime settings.
arXiv Machine Learning
Aug 12
A lower bound for stepsize-based acceleration of gradient descent
arXiv:2608. 10418v1 Announce Type: cross Abstract: Recent work has shown that, for smooth convex optimization, plain gradient descent can be accelerated from its textbook convergence rate of $O(T^{-1})$ (where $T$ denotes the number of iterations) to $O\big(T^{-\log_2(1+\sqrt{2})}\big)$ using carefully designed stepsize schedules alone, without resorting to momentum or other algorithmic modifications.
arXiv Machine Learning
Sep 10
The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives
arXiv:2609.08537v1 Announce Type: cross Abstract: We study the time-uniform convergence of the raw iterate of standard stochastic gradient descent (SGD) for unconstrained smooth convex objectives. We...
arXiv Machine Learning
Jun 4
Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
arXiv:2602. 05657v2 Announce Type: replace Abstract: The study of tail behaviour of SGD-induced processes has been attracting a lot of interest, due to offering strong guarantees with respect to individual runs of an algorithm.
arXiv Machine Learning
Jun 16
Tight Bounds for Logistic Regression with Large Stepsize Gradient Descent in Low Dimension
arXiv:2602. 12471v2 Announce Type: replace Abstract: We consider the optimization problem of minimizing the logistic loss with gradient descent to train a linear model for binary classification with separable data.