Convergence of Large-Stepsize SGD for the Cross-entropy Loss via Stochastic Lyapunov Stability
Read the original on arXiv Statistics ML →The paper presents convergence guarantees for Stochastic Gradient Descent (SGD) applied to the multiclass cross-entropy loss in both linear classifiers and two-layer neural networks. It shows that SGD dynamics can oscillate between an edge-of-stability regime dominated by curvature-driven oscillations and a stable regime where the expected loss decreases at a controlled rate. Despite these oscillations, the authors prove that SGD stabilizes the dynamics, ensuring that iterates return to stability within a fixed number of iterations and enabling convergence in the best-iterate sense even with large learning rates.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.