arXiv Machine Learning June 8, 2026 By Jonghyun Shin, Sejun Park

Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters

Read the original on arXiv Machine Learning →

arXiv:2606. 06934v1 Announce Type: new Abstract: We analyze generalization error, uniform stability, and uniform argument stability of gradient descent (GD) and stochastic gradient descent (SGD) over discrete parameter spaces, where each update involves deterministic or stochastic rounding.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

4d ago

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...

By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov

safety

arXiv AI

3d ago

Stochastic Gradient Descent with Momentum is Algorithmically Stable

arXiv:2605.28517v2 Announce Type: replace-cross Abstract: Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimizati...

By Yunwen Lei, Zimeng Wang, Xiaoming Yuan

arXiv Machine Learning

Jul 17

What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity

arXiv:2607. 14731v1 Announce Type: new Abstract: Local SGD, also known as Federated Averaging, is a widely used distributed optimization algorithm.

By Kumar Kshitij Patel, Rustem Islamov, Sebastian U Stich, Aurelien Lucchi, Eduard Gorbunov, Lingxiao Wang

arXiv Machine Learning

Sep 10

Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networks

arXiv:2609.06430v1 Announce Type: new Abstract: We study the identity straight-through estimator (STE) for training a two-layer binary-activation network with hinge loss from the perspective of Stati...

By Yiming Ying

efficiency

arXiv Machine Learning

Aug 12

A lower bound for stepsize-based acceleration of gradient descent

arXiv:2608. 10418v1 Announce Type: cross Abstract: Recent work has shown that, for smooth convex optimization, plain gradient descent can be accelerated from its textbook convergence rate of $O(T^{-1})$ (where $T$ denotes the number of iterations) to $O\big(T^{-\log_2(1+\sqrt{2})}\big)$ using carefully designed stepsize schedules alone, without resorting to momentum or other algorithmic modifications.

By Jianhao Ma, Yuxin Chen

llms benchmarks

arXiv Machine Learning

Jun 18

On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials

arXiv:2602. 14789v2 Announce Type: replace Abstract: The dynamical stability of the iterates during training plays a key role in determining the minima obtained by optimization algorithms.

By Rotem Mulayoff, Sebastian U. Stich

One email a morning, machine-written

Related stories

High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness

Stochastic Gradient Descent with Momentum is Algorithmically Stable

What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity

Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networks

A lower bound for stepsize-based acceleration of gradient descent

On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials