arXiv Machine Learning October 28, 2026 By Yuxing Peng, Zhiqing Tang, Weijia Jia

The Cost of an Unknown Horizon for the Last Iterate of Clipped SGD under Heavy-Tailed Noise

Read the original on arXiv Machine Learning →

arXiv:2610. 33381v2 Announce Type: new Abstract: Training is often stopped at a data-dependent time, so guarantees for the last iterate must hold at all times simultaneously.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 4

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization

arXiv:2602. 05657v2 Announce Type: replace Abstract: The study of tail behaviour of SGD-induced processes has been attracting a lot of interest, due to offering strong guarantees with respect to individual runs of an algorithm.

By Aleksandar Armacki, Dragana Bajovi\'c, Du\v{s}an Jakoveti\'c, Soummya Kar, Ali H. Sayed

arXiv Machine Learning

Oct 5

Can SGD Handle Heavy-Tailed Noise?

arXiv:2508.04860v2 Announce Type: replace-cross Abstract: Stochastic Gradient Descent (SGD) is a cornerstone of large-scale optimization, yet its theoretical behavior under heavy-tailed noise -- comm...

By Ilyas Fatkhullin, Florian H\"ubler, Guanghui Lan

reinforcement-learning

arXiv Machine Learning

Jul 30

Parameter-Free Dynamic Regret for Online Convex Optimization under Heavy-Tailed Noise

arXiv:2607. 27073v1 Announce Type: new Abstract: We study online convex optimization (OCO) in non-stationary environments under heavy-tailed noise, where the stochastic gradient oracle admits only a finite $p$-th central moment for some $p \in (1, 2]$.

By Vaneet Aggarwal

arXiv Machine Learning

Sep 30

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...

By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov

safety

arXiv Machine Learning

Sep 11

Silver Rate Is (Almost) Optimal for Gradient Descent

arXiv:2609. 09152v2 Announce Type: replace-cross Abstract: We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization.

By Yuhan Ye, Kaizhao Liu

arXiv Machine Learning

Sep 10

Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration

arXiv:2609.09152v1 Announce Type: cross Abstract: We study how far gradient descent (GD) can be accelerated by predetermined nonnegative stepsizes in smooth convex optimization. Writing $p_{\mathrm{s...

By Yuhan Ye, Kaizhao Liu

One email a morning, machine-written

Related stories

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization

Can SGD Handle Heavy-Tailed Noise?

Parameter-Free Dynamic Regret for Online Convex Optimization under Heavy-Tailed Noise

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

Silver Rate Is (Almost) Optimal for Gradient Descent

Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration