arXiv Machine Learning
Jun 2
Multigrade Neural Network Approximation
arXiv:2601. 16884v3 Announce Type: replace Abstract: We study multigrade deep learning (MGDL) as a principled framework for structured error refinement in deep neural networks.
Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions
arXiv:2610. 31264v1 Announce Type: new Abstract: We construct a continuous function on $[-1,1]^2$ for which the infimum of the $L^2$-approximation error over the class of ReLU networks with two hidden layers and architecture $(2,2,1,1)$ is not attained.
Related stories
arXiv Machine Learning
6d ago
Optimal approximation of smooth functions by ReLU neural networks
arXiv:2610.22693v1 Announce Type: cross Abstract: We study uniform approximation of the H\"older ball $\mathcal H_1^s([0,1]^D)$ by fully connected ReLU networks. Except in the one-dimensional Lipschi...
arXiv Machine Learning
Jul 21
Effects of width-dependent model hyperparameters and $\ell_2$-regularization on the loss landscape of two-layer ReLU networks
arXiv:2607. 16720v1 Announce Type: new Abstract: Understanding deep neural networks remains a central challenge in machine learning.
arXiv AI
Sep 24
Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon
The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.
arXiv Machine Learning
Jul 7
Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One
arXiv:2607. 04597v1 Announce Type: new Abstract: In this paper, we study the universal approximation property of residual neural networks, and obtain some new results.
arXiv Machine Learning
Aug 11
Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks
arXiv:2502. 11152v4 Announce Type: replace-cross Abstract: The optimization foundations of deep linear networks have recently received significant attention.
arXiv Machine Learning
Sep 22
Approximating Smooth Functionals with ReLU Networks
arXiv:2609.15355v2 Announce Type: replace-cross Abstract: We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by ReLU neural netw...
arXiv Machine Learning
Jun 24
Layer-wise Geometric Approximation Rates for Deep Networks
arXiv:2604. 20219v2 Announce Type: replace Abstract: Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for the final output and leaves the role of intermediate layers largely unclear.
arXiv Machine Learning
Oct 9
NP-Hardness of Minimizing Neurons in Two-Hidden-Layer ReLU Neural Networks
arXiv:2610. 11313v1 Announce Type: new Abstract: A fundamental question in neural network architecture optimization is whether the minimum hidden-neuron count required to approximate a target function within a prescribed tolerance can be computed efficiently.
arXiv Machine Learning
Oct 5
Theoretical Lower Bounds on the Robustness of Deep ReLU Networks
arXiv:2602.18674v2 Announce Type: replace Abstract: We present a theoretical study of the robustness of parameterized neural networks to random input perturbations. Specifically, we analyze local rob...
arXiv Machine Learning
Jul 7
Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse
arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.
arXiv Machine Learning
Oct 15
The minimal width of neural networks and immersions of real projective space
arXiv:2610.20229v1 Announce Type: cross Abstract: We investigate the minimal width of deep neural networks that can approximate all continuous functions from $\mathbb{R}^n$ to $\mathbb{R}^m$ in the t...