arXiv Machine Learning August 13, 2026 By Siqiao Mu, Diego Klabjan

Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness

Read the original on arXiv Machine Learning →

arXiv:2608. 11479v1 Announce Type: new Abstract: We establish convergence guarantees of gradient descent for general feedforward neural networks of arbitrary width or depth, with no special requirements on the initialization or dataset.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Sep 24

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon

The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.

By Hao Yu

efficiency safety

arXiv Machine Learning

Aug 12

On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities

arXiv:2605. 10775v2 Announce Type: replace-cross Abstract: A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity.

By Romain Petit, Clarice Poon, Gabriel Peyr\'e

arXiv AI

Jul 13

Principles of Lipschitz continuity in neural networks

arXiv:2602. 04078v2 Announce Type: replace-cross Abstract: Deep learning has achieved remarkable success across a wide range of domains, significantly expanding the frontiers of what is achievable in artificial intelligence.

By R\'ois\'in Luo

arXiv Machine Learning

Sep 10

A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay

arXiv:2609.07755v1 Announce Type: new Abstract: Understanding generalization remains a central challenge in machine learning because it requires jointly considering data, architecture, and training d...

By Yuqing Wang, Ioannis G. Kevrekidis, Mikhail Belkin

arXiv Machine Learning

Aug 11

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks

arXiv:2502. 11152v4 Announce Type: replace-cross Abstract: The optimization foundations of deep linear networks have recently received significant attention.

By Po Chen, Rujun Jiang, Peng Wang

arXiv Machine Learning

Sep 10

Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networks

arXiv:2609.06430v1 Announce Type: new Abstract: We study the identity straight-through estimator (STE) for training a two-layer binary-activation network with hinge loss from the perspective of Stati...

By Yiming Ying

efficiency

One email a morning, machine-written

Related stories

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon

On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities

Principles of Lipschitz continuity in neural networks

A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks

Stability and Generalization of Straight-Through Estimators for Training Two-Layer Quantized Neural Networks