arXiv Machine Learning September 10, 2026 By Yuqing Wang, Ioannis G. Kevrekidis, Mikhail Belkin

A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay

Read the original on arXiv Machine Learning →

The Flow has not summarised this story yet — read it at arXiv Machine Learning.

arXiv Machine Learning

Sep 10

The Dynamics of Generalization in Deep Learning

arXiv:2504.16450v4 Announce Type: replace Abstract: We derive a differential equation that governs the evolution of the generalization gap when a model is trained by gradient descent-based methods. T...

By Rubing Yang, Pratik Chaudhari

arXiv AI

Jun 2

The Geometry of Grokking: Norm Minimization on the Zero-Loss Manifold

arXiv:2511. 01938v3 Announce Type: replace-cross Abstract: Grokking is a puzzling phenomenon in neural networks where full generalization occurs only after a substantial delay following the complete memorization of the training data.

By Tiberiu Musat

arXiv Machine Learning

Jun 30

Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization

arXiv:2505. 22578v2 Announce Type: replace Abstract: The optimization of neural networks under weight decay remains poorly understood from a theoretical standpoint.

By Etienne Boursier, Matthew Bowditch, Matthias Englert, Ranko Lazic

arXiv AI

Jun 29

How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks

arXiv:2606. 28242v1 Announce Type: cross Abstract: Understanding how performance scales jointly with model size and data is a central problem in modern machine learning.

By Julius Girardin, Emanuele Troiani, Yizhou Xu, Vittorio Erba, Florent Krzakala, Lenka Zdeborov\'a

arXiv AI

Jun 8

Generalization in Deep Neural Networks: Minimax Rates for Gradient Methods

arXiv:2606. 06772v1 Announce Type: cross Abstract: Understanding the generalization performance of over-parameterized neural networks has become a central topic in deep learning theory.

By Junyu Zhou, Puyu Wang, Yunwen Lei, Marius Kloft, Yiming Ying

arXiv Machine Learning

Jun 17

A Dynamical Systems Perspective on the Analysis of Neural Networks

arXiv:2507. 05164v2 Announce Type: replace-cross Abstract: In this chapter, we utilize dynamical systems to analyze several aspects of machine learning algorithms.

By Dennis Chemnitz, Maximilian Engel, Christian Kuehn, Sara-Viola Kuntz

rag safety

One email a morning, machine-written

Related stories

The Dynamics of Generalization in Deep Learning

The Geometry of Grokking: Norm Minimization on the Zero-Loss Manifold

Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization

How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks

Generalization in Deep Neural Networks: Minimax Rates for Gradient Methods

A Dynamical Systems Perspective on the Analysis of Neural Networks