arXiv Machine Learning July 8, 2026 By Zhengdao Chen, Eric Vanden-Eijnden, Joan Bruna

A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks

Read the original on arXiv Machine Learning →

arXiv:2210. 16286v2 Announce Type: replace Abstract: To understand the training dynamics of neural networks, prior studies have considered the mean-field limit of two-layer neural networks as the width tends to infinity, establishing theoretical guarantees for its convergence under gradient flow training as well as approximation and generalization capabilities.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 12

On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities

arXiv:2605. 10775v2 Announce Type: replace-cross Abstract: A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity.

By Romain Petit, Clarice Poon, Gabriel Peyr\'e

arXiv Machine Learning

Jun 8

Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders

arXiv:2606. 07120v1 Announce Type: new Abstract: Autoencoders (AEs) learn low-dimensional representations by mapping data into a latent space while minimizing reconstruction error.

By Santanu Das, Ramyak Bilas, Pascal Esser, Satyaki Mukherjee

diffusion

arXiv Machine Learning

Jul 1

Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization

arXiv:2511. 21466v3 Announce Type: replace Abstract: We study Consensus-Based Optimization (CBO) for two-layer neural network training.

By William De Deyn, Michael Herty, Giovanni Samaey

arXiv Machine Learning

Jul 7

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.

By Shuang Liang, Tom Jacobs, Guido Mont\'ufar

safety

arXiv Machine Learning

Jul 8

Width-Robust Learnability in Mean-Field Bayesian Neural Networks

arXiv:2607. 05735v1 Announce Type: cross Abstract: Infinite-width limits are a standard way to reason about neural networks, but it is not automatic that the limiting learner has the same complexity-theoretic inductive bias as large finite networks.

By Dmitry Vaintrob, Kaarel H\"anni

safety

arXiv Machine Learning

Aug 11

Weak Correlations as the Underlying Principle for Linearization of Gradient-Based Learning Systems

arXiv:2401. 04013v2 Announce Type: replace Abstract: Deep learning models, such as wide neural networks, can be conceptualized as nonlinear dynamical physical systems characterized by a multitude of interacting degrees of freedom.

By Ori Shem-Ur, Yaron Oz

Related stories

On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities

Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders

Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

Width-Robust Learnability in Mean-Field Bayesian Neural Networks

Weak Correlations as the Underlying Principle for Linearization of Gradient-Based Learning Systems