arXiv Machine Learning
Stochastic Gradient Descent Ascent is Suboptimal for Nonconvex-PL Min-Max Games
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 3
Improved Gradient Descent Lower Bounds Beyond Nesterov
The paper investigates the limits of accelerating gradient descent (GD) using predetermined step sizes in smooth convex optimization. It establishes new lower bounds: an ≥·n−1.6342 non‑anytime bound and an ≥·n−1.2408 anytime bound, surpassing previous results. These findings also demonstrate a strict separation between convergence exponents achievable in non‑anytime versus anytime settings.
arXiv Machine Learning
Sep 10
How to Make the Gradient Mapping Small for Constrained Stochastic Min-Max Problems and Beyond
arXiv:2609.08380v1 Announce Type: cross Abstract: We study the stochastic first-order oracle complexity for constrained or regularized convex-concave min-max optimization and stochastic monotone vari...
arXiv Machine Learning
Oct 5
Can SGD Handle Heavy-Tailed Noise?
arXiv:2508.04860v2 Announce Type: replace-cross Abstract: Stochastic Gradient Descent (SGD) is a cornerstone of large-scale optimization, yet its theoretical behavior under heavy-tailed noise -- comm...
arXiv Machine Learning
Sep 21
Single-Loop Stochastic Projected Damped Extragradient Methods for Stochastic Nonconvex--(Strongly) Concave Minimax Optimization
The paper introduces single-loop stochastic projected damped extragradient (SPDE) and its variance-reduced variant (VR-SPDE) for stochastic nonconvex–(strongly) concave minimax problems. It provides SFO complexity bounds for achieving game stationarity and optimization stationarity, improving upon previous multi-loop methods while maintaining a single-loop structure. The results claim the best-known SFO complexities for these stationarity criteria among single-loop stochastic first‑order methods.
arXiv Machine Learning
Sep 29
Tight Stochastic Condition-Number Dependence in Nonconvex-Strongly-Concave Minimax Optimization
arXiv:2609. 30877v1 Announce Type: cross Abstract: We study whether the linear condition-number dependence in the stochastic complexity of SAPD+ is necessary for nonconvex-strongly-concave minimax optimization.
arXiv Machine Learning
Sep 10
Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration
arXiv:2609.09152v1 Announce Type: cross Abstract: We study how far gradient descent (GD) can be accelerated by predetermined nonnegative stepsizes in smooth convex optimization. Writing $p_{\mathrm{s...