arXiv Machine Learning June 16, 2026 By Bohao Ma, Junyu Zhang, Chuan He

Schattor: Schatten-family methods for deep learning optimization

Read the original on arXiv Machine Learning →

arXiv:2606. 15702v1 Announce Type: cross Abstract: Modern deep learning optimization features heterogeneous parameter structures, noisy gradients, and highly nonconvex landscapes, posing significant challenges for both algorithm design and theoretical analysis.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 31

The Convergence Behavior of Adam under Heavy-Tailed Noise

arXiv:2607. 27383v1 Announce Type: new Abstract: We establish the first convergence guarantees for the plain vector-form \emph{Adam} optimizer under heavy-tailed stochastic noise.

By Yijiang Pang

arXiv Statistics ML

Aug 26

Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise

arXiv:2506.04192v4 Announce Type: replace-cross Abstract: Stochastic Frank-Wolfe is a classical optimization method for solving constrained optimization problems. On the other hand, recent optimizers...

By Maria-Eleni Sfyraki, Jun-Kun Wang

arXiv Machine Learning

Jun 9

OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality

arXiv:2606. 08783v1 Announce Type: cross Abstract: Orthogonalized momentum updates, as used in Muon-style optimizers, have recently shown strong empirical stability in large-scale deep learning.

By Ganzhao Yuan

arXiv Machine Learning

Jun 2

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients

arXiv:2512. 02342v3 Announce Type: replace-cross Abstract: The stochastic Polyak step size (SPS) has proven to be a promising choice for stochastic gradient descent (SGD), delivering competitive performance relative to state-of-the-art methods on smooth convex and non-convex optimization problems, including deep neural network training.

By Dimitris Oikonomou, Nicolas Loizou

benchmarks

arXiv AI

Sep 10

The Role of Gradient Modification in Heavy-Tailed Nonconvex Stochastic Min-Max Optimization

arXiv:2609.06064v1 Announce Type: cross Abstract: Stochastic min-max optimization has attracted increasing attention due to its applications in modern machine learning, while existing theoretical stu...

By Tianxi Zhu, Yi Xu, Xiangyang Ji

arXiv Machine Learning

Jun 16

OptEMA: Adaptive Exponential Moving Average for Stochastic Optimization with Zero-Noise Optimality

arXiv:2603. 09923v4 Announce Type: replace Abstract: Exponential moving averages (EMAs) are a central component of widely used adaptive optimizers such as Adam.

By Ganzhao Yuan

One email a morning, machine-written

Related stories

The Convergence Behavior of Adam under Heavy-Tailed Noise

Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise

OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients

The Role of Gradient Modification in Heavy-Tailed Nonconvex Stochastic Min-Max Optimization

OptEMA: Adaptive Exponential Moving Average for Stochastic Optimization with Zero-Noise Optimality