arXiv:2607. 10589v1 Announce Type: cross Abstract: In contrast to most studies on neural network approximation theory that characterize results through a single parameter, such as the total number of network parameters, \cite{shen2020deep} pioneered the characterization of approximation rates as a joint function of the width parameter $N$ and the depth parameter $L$, thereby granting greater architectural flexibility.
By Yanming Lai, Defeng Sun, Yang Wang
arXiv:2610.22693v1 Announce Type: cross
Abstract: We study uniform approximation of the H\"older ball $\mathcal H_1^s([0,1]^D)$ by fully connected ReLU networks. Except in the one-dimensional Lipschi...
By Thomas Nagler
Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm.
arXiv:2609. 19937v1 Announce Type: cross Abstract: Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights.
By Xianjun Li, Yunfei Yang
arXiv:2407.11353v5 Announce Type: replace
Abstract: We study nonparametric regression using an over-parameterized two-layer neural network trained with algorithmic guarantees in this paper. We consid...
By Yingzhen Yang, Ping Li
arXiv:2610.02253v1 Announce Type: new
Abstract: The universal approximation property of dropout neural networks does not by itself describe the network size required for an accurate random realizatio...
By Jia-He Yao
arXiv:2608.23877v1 Announce Type: new
Abstract: We prove a depth hierarchy for ReLU neural networks in which every additional ReLU layer can save exponentially many neurons. For every $\ell\geq 3$, a...
By Itay Safran
An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input-dependent sparsity in the one-hidden-layer ReLU model of Awasthi et al.
arXiv:2609.09130v1 Announce Type: new
Abstract: An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input...
By Xiaoyu Li, Zhizhou Sha, Jiaojiao Jiang, Junbin Gao, Andi Han
A persistent empirical observation is that trained neural networks outperform their neural tangent kernel (NTK) limit on tasks with compositional structure, yet a quantitative account of $\textbf{when}$ and $\textbf{by how much}$ has been lacking. Working on the unit circle, we give such an account through a dichotomy between two complexity measures of the target: its $\textbf{Fourier complexity}$, which controls NTK kernel regression, and its $\textbf{architectural complexity}$, which controls learning over depth-$L$, width-$w$ ReLU networks with the variation norm of the weights bounded by $R$.
arXiv:2607. 06382v1 Announce Type: cross Abstract: A persistent empirical observation is that trained neural networks outperform their neural tangent kernel (NTK) limit on tasks with compositional structure, yet a quantitative account of $\textbf{when}$ and $\textbf{by how much}$ has been lacking.
By Arkaprabha Ganguli, Emil Constantinescu
The paper proves that several decision and approximation problems for ReLU neural networks are computationally hard. For any number of layers λ≥2, deciding whether a network’s output is positive (and thus whether it is surjective) is W[ℓ−1]-hard when parameterized by the input dimension d. In particular, for two-layer networks, the related geometric problem of zonotope non‑containment is W[1]-hard in the ambient dimension, and computing or approximating the Lp‑Lipschitz constant is NP‑hard and W[ℓ−1]-hard with respect to d. The results also show that these problems remain hard when parameterized by the number of layers for constant d, implying that naive enumeration algorithms running in n^{(ℓ−1)d}·poly(N) time are essentially optimal under the Exponential Time Hypothesis.
By Vincent Froese, Moritz Grillo, Christoph Hertrich, Moritz Stargalla