arXiv Machine Learning October 8, 2026 By Aleksandar Armacki, Haoyuan Cai, Ali H. Sayed

Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping

Read the original on arXiv Machine Learning →

arXiv:2610. 10527v1 Announce Type: cross Abstract: Heavy-tailed noise has been widely observed in modern machine learning, motivating the use of methods like gradient clipping and normalization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

agents

arXiv Machine Learning

Sep 30

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...

By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov

safety

arXiv Machine Learning

Jun 8

Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization

arXiv:2606. 07496v1 Announce Type: new Abstract: Decentralized stochastic optimization is a fundamental paradigm for large-scale learning over networks, where agents communicate only with their neighbors and no central coordinator is required.

By Ming Sun, Kun Yuan

agents

arXiv Machine Learning

Jun 2

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad

arXiv:2605. 18694v2 Announce Type: replace-cross Abstract: Many tasks in modern machine learning are observed to involve heavy-tailed gradient noise during the optimization process.

By Zijian Liu

arXiv Machine Learning

Sep 14

Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness

arXiv:2609. 12785v1 Announce Type: new Abstract: Classical convergence guarantees for stochastic gradient methods typically assume Lipschitz-smooth objectives and finite-variance gradient noise, both frequently violated in practice.

By Misbah Uz Zaman, Anirbit Mukherjee

arXiv Machine Learning

Oct 5

Can SGD Handle Heavy-Tailed Noise?

arXiv:2508.04860v2 Announce Type: replace-cross Abstract: Stochastic Gradient Descent (SGD) is a cornerstone of large-scale optimization, yet its theoretical behavior under heavy-tailed noise -- comm...

By Ilyas Fatkhullin, Florian H\"ubler, Guanghui Lan

reinforcement-learning

arXiv Machine Learning

Oct 5

Variance-reduced accelerated methods for decentralized stochastic double-regularized nonconvex strongly-concave minimax problems

arXiv:2307. 07113v2 Announce Type: replace-cross Abstract: In this paper, we consider the decentralized, stochastic nonconvex strongly-concave (NCSC) minimax problem with nonsmooth regularization terms on both primal and dual variables, wherein a network of $m$ computing agents collaborate via peer-to-peer communications.

By Gabriel Mancino-Ball, Muhammad Khan, Yangyang Xu

agents

One email a morning, machine-written

Related stories

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad

Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness

Can SGD Handle Heavy-Tailed Noise?

Variance-reduced accelerated methods for decentralized stochastic double-regularized nonconvex strongly-concave minimax problems