arXiv Machine Learning June 4, 2026 By Aleksandar Armacki, Dragana Bajovi\'c, Du\v{s}an Jakoveti\'c, Soummya Kar, Ali H. Sayed

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization

Read the original on arXiv Machine Learning →

arXiv:2602. 05657v2 Announce Type: replace Abstract: The study of tail behaviour of SGD-induced processes has been attracting a lot of interest, due to offering strong guarantees with respect to individual runs of an algorithm.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

4d ago

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...

By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov

safety

arXiv Machine Learning

Sep 14

Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness

arXiv:2609. 12785v1 Announce Type: new Abstract: Classical convergence guarantees for stochastic gradient methods typically assume Lipschitz-smooth objectives and finite-variance gradient noise, both frequently violated in practice.

By Misbah Uz Zaman, Anirbit Mukherjee

arXiv Machine Learning

Sep 1

Convergence rates for the RMSprop optimizer with full control of the hyperparameters

arXiv:2608.30382v1 Announce Type: new Abstract: Popular adaptive stochastic gradient descent (SGD) methods to train artificial intelligence (AI) systems include the RMSprop, the Adam, and the AdamW o...

By Steffen Dereich, Arnulf Jentzen

arXiv Machine Learning

Jun 2

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad

arXiv:2605. 18694v2 Announce Type: replace-cross Abstract: Many tasks in modern machine learning are observed to involve heavy-tailed gradient noise during the optimization process.

By Zijian Liu

arXiv Machine Learning

Jun 3

Bregman meets L\'evy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time

arXiv:2606. 03769v1 Announce Type: cross Abstract: We study the robustness of stochastic mirror descent (SMD) under heavy-tailed noise, focusing on whether the method retains its convergence guarantees when run with infinite-variance stochastic gradient input.

By Pierre-Louis Cauvin, Panayotis Mertikopoulos

arXiv AI

Sep 10

Parameter-Free Dynamic Regret under Heavy-Tailed Noise

arXiv:2607. 27073v2 Announce Type: replace-cross Abstract: We study online convex optimization with stochastic gradient noise whose conditional $p$-th central moment is bounded by $\sigma^p$, for an unknown $p\in(1,2]$.

By Vaneet Aggarwal

One email a morning, machine-written

Related stories

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness

Convergence rates for the RMSprop optimizer with full control of the hyperparameters

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad

Bregman meets L\'evy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time

Parameter-Free Dynamic Regret under Heavy-Tailed Noise