arXiv Machine Learning June 3, 2026 By Pierre-Louis Cauvin, Panayotis Mertikopoulos

Bregman meets L\'evy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time

Read the original on arXiv Machine Learning →

arXiv:2606. 03769v1 Announce Type: cross Abstract: We study the robustness of stochastic mirror descent (SMD) under heavy-tailed noise, focusing on whether the method retains its convergence guarantees when run with infinite-variance stochastic gradient input.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 2

In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise

arXiv:2606. 00520v1 Announce Type: cross Abstract: Many stochastic gradient methods are believed not to converge when the noise in stochastic gradients has only a finite $p$-th moment for $p\in\left(1,2\right)$, a setting known as the heavy-tailed noise assumption.

By Zijian Liu

arXiv Machine Learning

Jun 4

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization

arXiv:2602. 05657v2 Announce Type: replace Abstract: The study of tail behaviour of SGD-induced processes has been attracting a lot of interest, due to offering strong guarantees with respect to individual runs of an algorithm.

By Aleksandar Armacki, Dragana Bajovi\'c, Du\v{s}an Jakoveti\'c, Soummya Kar, Ali H. Sayed

arXiv Machine Learning

Sep 14

Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness

arXiv:2609. 12785v1 Announce Type: new Abstract: Classical convergence guarantees for stochastic gradient methods typically assume Lipschitz-smooth objectives and finite-variance gradient noise, both frequently violated in practice.

By Misbah Uz Zaman, Anirbit Mukherjee

arXiv Machine Learning

4d ago

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...

By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov

safety

arXiv Machine Learning

Jun 26

High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

arXiv:2606. 26316v1 Announce Type: new Abstract: We study first-order methods for smooth objectives satisfying the Polyak-\L{}ojasiewicz (PL) condition when gradient samples are generated by an exogenous Markov chain.

By Dhruv Sarkar, Aprameyo Chakrabartty, Vaneet Aggarwal

safety

arXiv Machine Learning

Aug 5

Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework

arXiv:2608. 03001v1 Announce Type: cross Abstract: Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation.

By Junwen Qiu, Bohao Ma, Andre Milzarek, Junyu Zhang

One email a morning, machine-written

Related stories

In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization

Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\"{o}lder Smoothness

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework