← Back to all news
arXiv Machine Learning By Puyu Wang, Jan Schuchardt, Nikita Kalinin, Marcio Monteiro, Junyu Zhou, Sophie Fellenz, Christoph Lampert, Marius Kloft

Optimization Risk Bounds for Kolmogorov-Arnold Networks Trained by DP-SGD with Correlated Noise

Read the original on arXiv Machine Learning →

The paper provides the first optimization risk bounds for two‑layer Kolmogorov‑Arnold Networks (KANs) trained with clipped mini‑batch differentially private stochastic gradient descent (DP‑SGD) that uses temporally correlated noise. The bounds explicitly capture the effects of temporal correlation, clipping, mini‑batch sampling, and network width, and show that correlation can reduce leading noise terms while the clipping threshold influences an effective step size. Experiments on synthetic data and MNIST confirm the theoretical predictions, and the authors extend the results to population risk guarantees via algorithmic stability, recovering several known special cases.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 14

Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise

The paper proves that stochastic gradient descent with gradient clipping and additive Gaussian noise (SGD‑CN) converges almost surely under smoothness and bounded noise assumptions, given standard decaying step sizes. The analysis extends to momentum variants such as the stochastic heavy ball and Nesterov's accelerated gradient, showing that careful energy constructions yield similar guarantees. These results provide stronger theoretical foundations for understanding the pathwise behaviour of clipped stochastic gradient methods in both convex and nonconvex regimes.

By Amartya Mukherjee, Jun Liu
safety
More like this →
arXiv Machine Learning
Jun 2

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients

arXiv:2512. 02342v3 Announce Type: replace-cross Abstract: The stochastic Polyak step size (SPS) has proven to be a promising choice for stochastic gradient descent (SGD), delivering competitive performance relative to state-of-the-art methods on smooth convex and non-convex optimization problems, including deep neural network training.

By Dimitris Oikonomou, Nicolas Loizou
benchmarks
More like this →
arXiv Machine Learning
Sep 14

High-Probability Convergence of SGD via Batched Updates

The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.

By Feng Zhu, Robert W. Heath Jr., Aritra Mitra
agents
More like this →