arXiv Machine Learning July 8, 2026 By Zhengdao Chen, Eric Vanden-Eijnden, Joan Bruna

A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks

Read the original on arXiv Machine Learning →

arXiv:2210. 16286v2 Announce Type: replace Abstract: To understand the training dynamics of neural networks, prior studies have considered the mean-field limit of two-layer neural networks as the width tends to infinity, establishing theoretical guarantees for its convergence under gradient flow training as well as approximation and generalization capabilities.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 12

On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities

arXiv:2605. 10775v2 Announce Type: replace-cross Abstract: A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity.

By Romain Petit, Clarice Poon, Gabriel Peyr\'e

arXiv Machine Learning

Jun 8

Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders

arXiv:2606. 07120v1 Announce Type: new Abstract: Autoencoders (AEs) learn low-dimensional representations by mapping data into a latent space while minimizing reconstruction error.

By Santanu Das, Ramyak Bilas, Pascal Esser, Satyaki Mukherjee

diffusion

arXiv Machine Learning

Jul 1

Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization

arXiv:2511. 21466v3 Announce Type: replace Abstract: We study Consensus-Based Optimization (CBO) for two-layer neural network training.

By William De Deyn, Michael Herty, Giovanni Samaey

arXiv Machine Learning

Jul 7

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.

By Shuang Liang, Tom Jacobs, Guido Mont\'ufar

safety

arXiv Machine Learning

Jul 8

Width-Robust Learnability in Mean-Field Bayesian Neural Networks

arXiv:2607. 05735v1 Announce Type: cross Abstract: Infinite-width limits are a standard way to reason about neural networks, but it is not automatic that the limiting learner has the same complexity-theoretic inductive bias as large finite networks.

By Dmitry Vaintrob, Kaarel H\"anni

safety

arXiv AI

Sep 24

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon

The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.

By Hao Yu

efficiency safety

One email a morning, machine-written

Related stories

On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities

Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders

Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

Width-Robust Learnability in Mean-Field Bayesian Neural Networks

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon