arXiv Machine Learning August 12, 2026 By Romain Petit, Clarice Poon, Gabriel Peyr\'e

On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities

Read the original on arXiv Machine Learning →

arXiv:2605. 10775v2 Announce Type: replace-cross Abstract: A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 8

A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks

arXiv:2210. 16286v2 Announce Type: replace Abstract: To understand the training dynamics of neural networks, prior studies have considered the mean-field limit of two-layer neural networks as the width tends to infinity, establishing theoretical guarantees for its convergence under gradient flow training as well as approximation and generalization capabilities.

By Zhengdao Chen, Eric Vanden-Eijnden, Joan Bruna

arXiv Machine Learning

6d ago

Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness

arXiv:2608. 11479v1 Announce Type: new Abstract: We establish convergence guarantees of gradient descent for general feedforward neural networks of arbitrary width or depth, with no special requirements on the initialization or dataset.

By Siqiao Mu, Diego Klabjan

arXiv Machine Learning

Jun 4

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.

By Berk Tinaz, Changzhi Xie, Mahdi Soltanolkotabi

safety

arXiv Machine Learning

Jul 13

How are linear representations learned? Exact solutions to the dynamics of abstraction

arXiv:2607. 08843v1 Announce Type: new Abstract: In artificial and biological neural networks, concepts are often encoded as consistent linear directions in representation space.

By William W. Yang, Andrew M. Saxe, Peter E. Latham

llms safety

arXiv Machine Learning

Aug 6

Towards Understanding Gradient Flow Dynamics of Homogeneous Neural Networks Beyond the Origin

arXiv:2502. 15952v3 Announce Type: replace Abstract: Recent works exploring the training dynamics of homogeneous neural network weights under gradient flow with small initialization have established that in the early stages of training, the weights remain small and near the origin, but converge in direction.

By Akshay Kumar, Jarvis Haupt

efficiency

arXiv Machine Learning

Jun 17

A Dynamical Systems Perspective on the Analysis of Neural Networks

arXiv:2507. 05164v2 Announce Type: replace-cross Abstract: In this chapter, we utilize dynamical systems to analyze several aspects of machine learning algorithms.

By Dennis Chemnitz, Maximilian Engel, Christian Kuehn, Sara-Viola Kuntz

rag safety

Related stories

A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks

Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

How are linear representations learned? Exact solutions to the dynamics of abstraction

Towards Understanding Gradient Flow Dynamics of Homogeneous Neural Networks Beyond the Origin

A Dynamical Systems Perspective on the Analysis of Neural Networks