arXiv Machine Learning June 24, 2026 By Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed

Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum

Read the original on arXiv Machine Learning →

arXiv:2406. 13041v3 Announce Type: replace Abstract: Lower-bound analyses for nonconvex strongly-concave minimax optimization problems have shown that stochastic first-order algorithms require at least $\mathcal{O}(\varepsilon^{-4})$ sample complexity to find an $\varepsilon$-stationary point.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

safety

arXiv AI

Sep 10

The Role of Gradient Modification in Heavy-Tailed Nonconvex Stochastic Min-Max Optimization

arXiv:2609.06064v1 Announce Type: cross Abstract: Stochastic min-max optimization has attracted increasing attention due to its applications in modern machine learning, while existing theoretical stu...

By Tianxi Zhu, Yi Xu, Xiangyang Ji

arXiv AI

3d ago

Stochastic Gradient Descent with Momentum is Algorithmically Stable

arXiv:2605.28517v2 Announce Type: replace-cross Abstract: Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimizati...

By Yunwen Lei, Zimeng Wang, Xiaoming Yuan

arXiv Machine Learning

Jul 2

Towards Weaker Variance Assumptions for Stochastic Optimization

arXiv:2504. 09951v2 Announce Type: replace-cross Abstract: We revisit a classical assumption for analyzing stochastic gradient algorithms where the squared norm of the stochastic subgradient (or the variance for smooth problems) is allowed to grow as fast as the squared norm of the optimization variable.

By Ahmet Alacaoglu, Yura Malitsky, Stephen J. Wright

arXiv Machine Learning

Jul 10

Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization

arXiv:2607. 08104v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is a cornerstone of modern optimization.

By Ryusei Yamada, Naoki Sato, Hideaki Iiduka

arXiv Machine Learning

Jun 2

In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise

arXiv:2606. 00520v1 Announce Type: cross Abstract: Many stochastic gradient methods are believed not to converge when the noise in stochastic gradients has only a finite $p$-th moment for $p\in\left(1,2\right)$, a setting known as the heavy-tailed noise assumption.

By Zijian Liu

arXiv Machine Learning

Jun 9

OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality

arXiv:2606. 08783v1 Announce Type: cross Abstract: Orthogonalized momentum updates, as used in Muon-style optimizers, have recently shown strong empirical stability in large-scale deep learning.

By Ganzhao Yuan

One email a morning, machine-written

Related stories

The Role of Gradient Modification in Heavy-Tailed Nonconvex Stochastic Min-Max Optimization

Stochastic Gradient Descent with Momentum is Algorithmically Stable

Towards Weaker Variance Assumptions for Stochastic Optimization

Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization

In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise

OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality