The paper proposes a conjecture that composing a fixed number of distinct nonconstant polynomials with a generic high‑degree polynomial produces linearly independent polynomials, extending Newman–Slater’s theorem. The authors prove the conjecture for two polynomials and for any number when the degrees are bounded, and they show how these results explain the parameter symmetries of deep fully connected neural networks with generic polynomial activations. In particular, for architectures with layer‑specific activations of increasing degree, the conjecture’s proven cases fully characterize the parameter sets that yield the same end‑to‑end network function, and it also resolves the identifiability of shallow polynomial networks.
By Kathl\'en Kohn, Giovanni Luca Marchetti, Alex Massarenti, Massimiliano Mella
arXiv:2607. 20811v1 Announce Type: new Abstract: In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions.
By Cornelius Brand, Robert Ganian, Mathis Rocton
arXiv:2410. 00722v3 Announce Type: replace Abstract: We study convolutional neural networks with monomial activation functions.
By Vahid Shahverdi, Giovanni Luca Marchetti, Kathl\'en Kohn
arXiv:2605. 29823v2 Announce Type: replace Abstract: Deep networks often exhibit a preference for "simple" solutions, and such a simplicity bias is widely believed to play a key role in generalization.
By Tianren Zhang, Xiangxin Li, Minghao Xiao, Guanyu Chen, Feng Chen
In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions. Indeed, while there has been a number of very recent results that establish ever-tighter lower bounds for the problem under linear and ReLU activation functions, less progress has been made towards the identification of novel polynomial-time tractable network architectures.
arXiv:2606. 28464v1 Announce Type: new Abstract: In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture.
By Kathl\'en Kohn, Giovanni Luca Marchetti, Farhan Shabir, Vahid Shahverdi, Weisheng Wang
arXiv:2511. 19703v2 Announce Type: replace-cross Abstract: We study the dimension and identifiability of neurovarieties associated to polynomial neural networks.
By A. Massarenti, M. Mella
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
By Yulia Alexandr, Guido Mont\'ufar
arXiv:2609. 28500v1 Announce Type: cross Abstract: We study exact representability by bias-free shallow polynomial neural networks using algebraic geometry.
By Sepehr Akbari, Shahrzad Jamshidi
arXiv:2607. 08370v1 Announce Type: cross Abstract: We derive bounds for the volume of tubular neighbourhoods of smooth Pfaffian hypersurfaces, generalising known results for algebraic varieties.
By Paul Lezeau, Martin Lotz
The paper demonstrates that tree tensor networks (TTNs) can encode arbitrary read‑once Boolean formulas, yielding polynomial‑size targets that are hard for gradient descent to learn in polynomial time, yet their loss landscapes are conditionally benign: every minimum‑norm local minimum is global. This shows that bad local minima are not the source of learning difficulty in TTNs; instead, high‑order degenerate saddle points caused by rank‑deficiency can impede learning. A case study on the parity function illustrates how TTNs can link landscape geometry to computational hardness.
By Zach Furman, Stephan W\"aldchen, Yangda Bei, Liam Hodgkinson
arXiv:2510. 04500v3 Announce Type: replace Abstract: This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance.
By Linghao Kong, Inimai Subramanian, Yonadav Shavit, Micah Adler, Dan Alistarh, Nir Shavit