arXiv Machine Learning July 30, 2026 By Junyu Zhou, Puyu Wang, Dennis Wagner, Yunwen Lei, Marius Kloft, Yiming Ying

Minimax-Optimal Generalization Bounds for Smooth Deep Neural Networks Trained by (Stochastic) Gradient Descent

Read the original on arXiv Machine Learning →

arXiv:2606. 06772v2 Announce Type: replace-cross Abstract: Characterizing the optimization dynamics and statistical performance of over-parameterized deep neural networks (DNNs) remains a central challenge in understanding the remarkable success of deep learning.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 8

Generalization in Deep Neural Networks: Minimax Rates for Gradient Methods

arXiv:2606. 06772v1 Announce Type: cross Abstract: Understanding the generalization performance of over-parameterized neural networks has become a central topic in deep learning theory.

By Junyu Zhou, Puyu Wang, Yunwen Lei, Marius Kloft, Yiming Ying

arXiv AI

Jun 8

Optimal Rates for Generalization of Gradient Descent Methods with Deep Neural Networks

arXiv:2606. 06764v1 Announce Type: cross Abstract: Recent progress has been made in understanding the statistical generalization performance of gradient descent methods for overparameterized neural networks within the neural tangent kernel (NTK) regime.

By Junyu Zhou, Puyu Wang, Yunwen Lei, Yiming Ying, Ding-Xuan Zhou

arXiv Machine Learning

Jun 3

Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification

arXiv:2510. 02779v4 Announce Type: replace Abstract: Recent advances have significantly improved our understanding of the generalization performance of gradient descent (GD) methods in deep neural networks.

By Yuanfan Li, Yunwen Lei, Zheng-Chu Guo, Yiming Ying

arXiv Machine Learning

Jun 5

Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations

arXiv:2606. 05599v1 Announce Type: new Abstract: This paper establishes a theoretical framework for the uniform convergence of smoothly activated deep neural network (DNN) estimators.

By Yizhe Ding, Runze Li, Jia Liu, Lingzhou Xue

arXiv Machine Learning

Jul 7

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

arXiv:2406. 14340v2 Announce Type: replace-cross Abstract: The standard stochastic gradient descent (SGD) optimization method, as well as adaptive methods such as the Adam optimizer fail to converge if the learning rates do not converge to zero (particularly, in the situation of constant learning rates).

By Steffen Dereich, Arnulf Jentzen, Adrian Riekert

arXiv Machine Learning

Aug 11

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

arXiv:2608. 09523v1 Announce Type: new Abstract: Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success.

By Binchuan Qi

safety

Related stories

Generalization in Deep Neural Networks: Minimax Rates for Gradient Methods

Optimal Rates for Generalization of Gradient Descent Methods with Deep Neural Networks

Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification

Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks