In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions. Indeed, while there has been a number of very recent results that establish ever-tighter lower bounds for the problem under linear and ReLU activation functions, less progress has been made towards the identification of novel polynomial-time tractable network architectures.
arXiv:2602.13106v2 Announce Type: replace-cross
Abstract: In recent years, there has been growing interest in understanding neural architectures' ability to learn to execute discrete algorithms, a li...
By Solveig Wittig, Antonis Vasileiou, Robert R. Nerem, Timo Stoll, Floris Geerts, Yusu Wang, Christopher Morris
The paper proposes a conjecture that composing a fixed number of distinct nonconstant polynomials with a generic high‑degree polynomial produces linearly independent polynomials, extending Newman–Slater’s theorem. The authors prove the conjecture for two polynomials and for any number when the degrees are bounded, and they show how these results explain the parameter symmetries of deep fully connected neural networks with generic polynomial activations. In particular, for architectures with layer‑specific activations of increasing degree, the conjecture’s proven cases fully characterize the parameter sets that yield the same end‑to‑end network function, and it also resolves the identifiability of shallow polynomial networks.
By Kathl\'en Kohn, Giovanni Luca Marchetti, Alex Massarenti, Massimiliano Mella
arXiv:2606. 26705v1 Announce Type: cross Abstract: Feedforward neural network (NN) expressivity is typically studied by emulating optimal basis-expansion schemes.
By Anastasis Kratsios, Simone Brugiapaglia, Bum Jun Kim, Gregory Cousins, Haitz S\'aez de Oc\'ariz Borde
arXiv:2608.24865v1 Announce Type: cross
Abstract: Lipschitz constants are a standard way to quantify the sensitivity of neural networks to small input perturbations, but computing them is difficult e...
By Aritra Das, Vincent Froese, Moritz Grillo, Debayan Gupta, Christoph Hertrich, Tharrshann Jayan Logarajah, Georg Loho, Mihir More, Moritz Stargalla
The paper proves that several decision and approximation problems for ReLU neural networks are computationally hard. For any number of layers λ≥2, deciding whether a network’s output is positive (and thus whether it is surjective) is W[ℓ−1]-hard when parameterized by the input dimension d. In particular, for two-layer networks, the related geometric problem of zonotope non‑containment is W[1]-hard in the ambient dimension, and computing or approximating the Lp‑Lipschitz constant is NP‑hard and W[ℓ−1]-hard with respect to d. The results also show that these problems remain hard when parameterized by the number of layers for constant d, implying that naive enumeration algorithms running in n^{(ℓ−1)d}·poly(N) time are essentially optimal under the Exponential Time Hypothesis.
By Vincent Froese, Moritz Grillo, Christoph Hertrich, Moritz Stargalla
arXiv:2607. 05546v1 Announce Type: cross Abstract: We develop a unified function space theory of deep fully connected neural networks.
By Julia Nakhleh, Robert D. Nowak
arXiv:2602. 06737v2 Announce Type: replace Abstract: We present a generalized framework for the range verification of neural networks featuring non-linear activation functions.
By Noah Schwartz, Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur, Tuhin Sahai, Susmit Jha
arXiv:2608. 09707v1 Announce Type: cross Abstract: Embedding trained neural networks as surrogates within optimisation problems is an established practice in operations research.
By Yu Liu, Jan Kronqvist, Fabricio Oliveira
arXiv:2607. 01266v1 Announce Type: cross Abstract: We study binary classification problems whose decision sets are given by definable sets in o-minimal expansions of the real field.
By Clemens Kinn, Philipp Petersen
arXiv:2510. 10101v4 Announce Type: replace Abstract: Understanding the interplay between generalization, expressivity, and the geometry of the input space is a central challenge in graph learning.
By Martin Carrasco, Caio F. Deberaldini Netto, Vahan A. Martirosyan, Ehimare Okoyomon, Caterina Graziani
arXiv:2610.00420v1 Announce Type: new
Abstract: A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such...
By Yuxin Ma, Adir Dayan, Yam Eitan, Haggai Maron, Soledad Villar