← Back to all news
arXiv Machine Learning By Yaxin Yu, Long Chen, Minfu Feng

SHANG++: Robust Stochastic Acceleration under Multiplicative Noise

Read the original on arXiv Machine Learning →

The paper introduces SHANG++—an accelerated stochastic gradient descent algorithm designed to be robust under multiplicative noise scaling (MNS). Building on a semi‑implicit discretization called SHANG, SHANG++ adds a damping correction that improves stability and convergence for both convex and strongly convex objectives. Experiments on convex problems and deep learning tasks, including a noise‑robust test on ResNet‑34, show that SHANG++ consistently outperforms existing accelerated methods with minimal parameter sensitivity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 2

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients

arXiv:2512. 02342v3 Announce Type: replace-cross Abstract: The stochastic Polyak step size (SPS) has proven to be a promising choice for stochastic gradient descent (SGD), delivering competitive performance relative to state-of-the-art methods on smooth convex and non-convex optimization problems, including deep neural network training.

By Dimitris Oikonomou, Nicolas Loizou
benchmarks
More like this →
arXiv Machine Learning
3d ago

Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise

The paper proves that stochastic gradient descent with gradient clipping and additive Gaussian noise (SGD‑CN) converges almost surely under smoothness and bounded noise assumptions, given standard decaying step sizes. The analysis extends to momentum variants such as the stochastic heavy ball and Nesterov's accelerated gradient, showing that careful energy constructions yield similar guarantees. These results provide stronger theoretical foundations for understanding the pathwise behaviour of clipped stochastic gradient methods in both convex and nonconvex regimes.

By Amartya Mukherjee, Jun Liu
safety
More like this →