Stochastic Gradient Descent: A Nonlinear Time Series Perspective
Read the original on arXiv Machine Learning →This paper studies the statistical properties of Stochastic Gradient Descent (SGD) with a constant step size using the framework of iterated random functions. It demonstrates convergence in Euclidean distance via Geometric Moment Contraction, establishes quenched central limit theorems, invariance principles, and α-stable convergence for averaged SGD regardless of initialization, and explores the effect of step sizes through derivative processes. The authors further develop a Richardson-Romberg extrapolation with multiple step sizes, propose a bias-reduced variant, and introduce an online inference method that yields empirical confidence intervals with coverage approaching nominal levels as sample size increases.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.