The paper proves that several decision and approximation problems for ReLU neural networks are computationally hard. For any number of layers λ≥2, deciding whether a network’s output is positive (and thus whether it is surjective) is W[ℓ−1]-hard when parameterized by the input dimension d. In particular, for two-layer networks, the related geometric problem of zonotope non‑containment is W[1]-hard in the ambient dimension, and computing or approximating the Lp‑Lipschitz constant is NP‑hard and W[ℓ−1]-hard with respect to d. The results also show that these problems remain hard when parameterized by the number of layers for constant d, implying that naive enumeration algorithms running in n^{(ℓ−1)d}·poly(N) time are essentially optimal under the Exponential Time Hypothesis.
By Vincent Froese, Moritz Grillo, Christoph Hertrich, Moritz Stargalla
arXiv:2607. 07778v1 Announce Type: new Abstract: Bubeck, Li and Nagaraj conjectured that, for generic data, any two-layer neural network with $m$ neurons that fits $n$ noisy labels must have Lipschitz constant at least of order $\sqrt{n/m}$, with no restriction on the size of the weights.
By Yitzchak Shmalo
arXiv:2609. 03129v1 Announce Type: cross Abstract: Several classical machine-learning methods, such as KRRs and SVRs, are both computationally and analytically tractable since their estimators either admit closed-form expressions or are obtained by minimizing convex training objectives; neither feature is generally available for deep neural networks.
By Ruiyang Hong, Hrad Ghoukasian, Anastasis Kratsios
arXiv:2607. 20811v1 Announce Type: new Abstract: In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions.
By Cornelius Brand, Robert Ganian, Mathis Rocton
arXiv:2608. 25221v1 Announce Type: new Abstract: We study exact representations of $\mathrm{MAX}_N(x)=\max{x_1,\ldots,x_N}$ using two-hidden-layer ReLU neural networks.
By Zhimao Wang, Amitabh Basu
In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions. Indeed, while there has been a number of very recent results that establish ever-tighter lower bounds for the problem under linear and ReLU activation functions, less progress has been made towards the identification of novel polynomial-time tractable network architectures.
arXiv:2608.31157v1 Announce Type: new
Abstract: Many parameter-efficient methods generate the parameters of a large neural network from a low-dimensional latent representation. Given an architecture...
By Shijun Zhang
arXiv:2607. 06781v1 Announce Type: new Abstract: In this work, we investigate the fixed-architecture neural network approximation with explicit parameter bounds and elementary activations.
By Feng-Lei Fan, Ze-Yu Li, Chen-Yu Wang, Jian-Jun Wang
arXiv:2605. 18528v2 Announce Type: replace-cross Abstract: A growing lesson from neural network optimization is that optimizer design should respect how the model is parametrized.
By Jiayu Zhang, Tianyi Lin
arXiv:2607. 04597v1 Announce Type: new Abstract: In this paper, we study the universal approximation property of residual neural networks, and obtain some new results.
By Qi Zhou, Xuan Zhou, Xiao-Song Yang
arXiv:2605. 31152v2 Announce Type: replace-cross Abstract: This paper studies how efficiently deep ReLU neural networks can approximate and learn smooth functions.
By Yunfei Yang, Jun Fan
arXiv:2607. 21651v1 Announce Type: new Abstract: We prove that the maximum of $n$ real numbers is exactly representable by a ReLU network with two hidden layers for every $n\le 10$.
By Kilian Rue{\ss}, Gennadiy Averkov, Florestan Brunck, Moritz Grillo, Christoph Hertrich, Georg Loho, Jack Stade, Moritz Stargalla, Matthew Sun, Martin Winter