arXiv Machine Learning June 16, 2026 By Huan Li, Yiming Dong, Zhouchen Lin

Convergence Rate Analysis of the AdamW-style Shampoo: Unifying One-Sided and Two-Sided Preconditioning

Read the original on arXiv Machine Learning →

arXiv:2601. 07326v4 Announce Type: replace-cross Abstract: This paper studies AdamW-style Shampoo, an effective variant of the classical Shampoo that won the external tuning track of the AlgoPerf neural network training competition.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 12

Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime

arXiv:2603. 10485v3 Announce Type: replace-cross Abstract: In this work, we study the convergence properties of the Dual Space Preconditioned Gradient Descent, encompassing optimizers such as Normalized Gradient Descent and Gradient Clipping.

By Reza Ghane, Danil Akhtiamov, Babak Hassibi

safety

arXiv Machine Learning

Jun 26

Gradient Testing and Estimation by Comparisons

arXiv:2405. 11454v3 Announce Type: replace Abstract: We study gradient testing and gradient estimation of smooth functions using only a comparison oracle that, given two points, indicates which one has the larger function value.

By Xiwen Tao, Chenyi Zhang, Helin Wang, Yexin Zhang, Tongyang Li

arXiv Machine Learning

Aug 4

Improved convergence rate of kNN graph Laplacians: differentiable self-tuned affinity

arXiv:2410. 23212v3 Announce Type: replace-cross Abstract: In graph-based data analysis, $k$-nearest neighbor ($k$NN) graphs are widely used due to their adaptivity to local data densities.

By Xiuyuan Cheng, Yixuan Tan, Nan Wu

safety

arXiv Machine Learning

Sep 1

Convergence rates for the RMSprop optimizer with full control of the hyperparameters

arXiv:2608.30382v1 Announce Type: new Abstract: Popular adaptive stochastic gradient descent (SGD) methods to train artificial intelligence (AI) systems include the RMSprop, the Adam, and the AdamW o...

By Steffen Dereich, Arnulf Jentzen

arXiv Machine Learning

Sep 25

Precise Convergence Speed of Clipped SGD

arXiv:2609. 29458v1 Announce Type: new Abstract: We present a tightened convergence analysis of clipped gradient descent on $(L_0, L_1)$-smooth functions, with quantitative constants.

By David A. R. Robin

safety

arXiv Machine Learning

Sep 7

Centered Permutation Prefixes for SGD with Random Reshuffling: Sharp Rates, H\"older Geometry, and Composite Proximal Extensions

arXiv:2609. 04578v1 Announce Type: cross Abstract: We study stochastic gradient descent with random reshuffling for finite sums \[ F(x)=\frac1n\sum_{i=1}^n f_i(x).

By Jiaxiang Li

One email a morning, machine-written

Related stories

Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime

Gradient Testing and Estimation by Comparisons

Improved convergence rate of kNN graph Laplacians: differentiable self-tuned affinity

Convergence rates for the RMSprop optimizer with full control of the hyperparameters

Precise Convergence Speed of Clipped SGD

Centered Permutation Prefixes for SGD with Random Reshuffling: Sharp Rates, H\"older Geometry, and Composite Proximal Extensions