← Back to all news
arXiv Machine Learning By Yuhan Ye, Kaizhao Liu

Improved Gradient Descent Lower Bounds Beyond Nesterov

Read the original on arXiv Machine Learning →

The paper investigates the limits of accelerating gradient descent (GD) using predetermined step sizes in smooth convex optimization. It establishes new lower bounds: an ≥·n−1.6342 non‑anytime bound and an ≥·n−1.2408 anytime bound, surpassing previous results. These findings also demonstrate a strict separation between convergence exponents achievable in non‑anytime versus anytime settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 12

A lower bound for stepsize-based acceleration of gradient descent

arXiv:2608. 10418v1 Announce Type: cross Abstract: Recent work has shown that, for smooth convex optimization, plain gradient descent can be accelerated from its textbook convergence rate of $O(T^{-1})$ (where $T$ denotes the number of iterations) to $O\big(T^{-\log_2(1+\sqrt{2})}\big)$ using carefully designed stepsize schedules alone, without resorting to momentum or other algorithmic modifications.

By Jianhao Ma, Yuxin Chen
llmsbenchmarks
More like this →
arXiv Machine Learning
Oct 5

Near-Optimal Convex Optimization with Lazy Second-Order Oracles

arXiv:2610. 03222v1 Announce Type: cross Abstract: This paper studies the complexity of convex optimization using lazy second-order oracles (Doikov, Chayti, and Jaggi, ICML 2023), where an algorithm queries gradients every iteration and Hessians once per $m$ iterations.

By Xinliang Zhang, Lesi Chen, Chengchang Liu, Jingzhao Zhang
More like this →