arXiv:2608. 10418v1 Announce Type: cross Abstract: Recent work has shown that, for smooth convex optimization, plain gradient descent can be accelerated from its textbook convergence rate of $O(T^{-1})$ (where $T$ denotes the number of iterations) to $O\big(T^{-\log_2(1+\sqrt{2})}\big)$ using carefully designed stepsize schedules alone, without resorting to momentum or other algorithmic modifications.
By Jianhao Ma, Yuxin Chen
arXiv:2610.19444v1 Announce Type: cross
Abstract: Random reshuffling stochastic gradient descent (RR-SGD) is widely used for finite-sum optimization in machine learning. For convex $L$-smooth compone...
By Zijun Chen, Zihan Zhang
The paper establishes the first optimal last‑iterate convergence rates, in high probability, for Clipped‑SGD when the gradient noise is heavy‑tailed. It addresses the gap left by earlier results that required strong noise assumptions, such as exponentially decaying tails. By proving these rates, the authors provide theoretical support for the practical effectiveness of gradient clipping in stochastic convex optimization.
By Zijian Liu
arXiv:2609.08537v1 Announce Type: cross
Abstract: We study the time-uniform convergence of the raw iterate of standard stochastic gradient descent (SGD) for unconstrained smooth convex objectives. We...
By Ruijie Li, Kang Chen, Tianyu Wang
arXiv:2606. 24879v1 Announce Type: cross Abstract: We study the last iterate of the stochastic subgradient method for one-dimensional convex Lipschitz objectives.
By Guglielmo Beretta, Tommaso Cesari, Roberto Colomboni, Andrea Paudice
arXiv:2606. 01764v1 Announce Type: cross Abstract: We revisit the convergence guarantees of the Extragradient (EG) method for unconstrained biaffine min-max optimization.
By Yue Wu, Weiqiang Zheng, Yang Cai, Haipeng Luo
arXiv:2608. 25551v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory.
By Liviu Aolaritei, Lucas L\'evy, Francis Bach, Michael I. Jordan
arXiv:2505.20817v3 Announce Type: replace-cross
Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov
arXiv:2505. 01423v2 Announce Type: replace-cross Abstract: Efficient computation of min-max problems is a central question in optimization, learning, games, and control.
By Henry Shugart, Jason M. Altschuler
This paper studies the statistical properties of Stochastic Gradient Descent (SGD) with a constant step size using the framework of iterated random functions. It demonstrates convergence in Euclidean distance via Geometric Moment Contraction, establishes quenched central limit theorems, invariance principles, and α-stable convergence for averaged SGD regardless of initialization, and explores the effect of step sizes through derivative processes. The authors further develop a Richardson-Romberg extrapolation with multiple step sizes, propose a bias-reduced variant, and introduce an online inference method that yields empirical confidence intervals with coverage approaching nominal levels as sample size increases.
By Jiaqi Li, Zhipeng Lou, Stefan Richter, Wei Biao Wu
arXiv:2606. 21528v2 Announce Type: replace-cross Abstract: We study first-order methods for solving monotone variational inequalities arising in min-max optimization.
By Motahareh Sohrabi, Jianxin You, Simon Lacoste-Julien, Eduard Gorbunov, Gauthier Gidel
arXiv:2609.09152v1 Announce Type: cross
Abstract: We study how far gradient descent (GD) can be accelerated by predetermined nonnegative stepsizes in smooth convex optimization. Writing $p_{\mathrm{s...
By Yuhan Ye, Kaizhao Liu