The paper proves the weak composition‑independence conjectures of Kohn, Marchetti, Massarenti and Mella by employing deformations of rational curves and a single kernel of the linearized equation, with Riemann‑Roch providing a contradiction from tangent fields with controlled poles. This result establishes that a neural network using a general polynomial activation function of sufficiently large degree is identifiable.
By Xiaotian Chang, Yacoub Hendi
arXiv:2511. 19703v2 Announce Type: replace-cross Abstract: We study the dimension and identifiability of neurovarieties associated to polynomial neural networks.
By A. Massarenti, M. Mella
The paper introduces Shallow Universal Polynomial Networks (SUPNs), a single‑hidden‑layer architecture that replaces standard activations with learnable multivariate polynomials, thereby increasing expressivity per unit. SUPNs achieve universal approximation by either widening the network or increasing polynomial complexity, and the authors prove convergence rates and quasi‑optimal parameter choices. Empirical tests across a wide range of smooth, nonsmooth, and discontinuous functions in up to ten dimensions show that SUPNs often outperform deep neural networks and Kolmogorov‑Arnold networks in accuracy and variability while requiring fewer parameters and lower training cost.
By Zachary Morrow, Michael Penwarden, Brian Chen, Aurya Javeed, Akil Narayan, John D. Jakeman
arXiv:2607. 20811v1 Announce Type: new Abstract: In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions.
By Cornelius Brand, Robert Ganian, Mathis Rocton
arXiv:2606. 28464v1 Announce Type: new Abstract: In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture.
By Kathl\'en Kohn, Giovanni Luca Marchetti, Farhan Shabir, Vahid Shahverdi, Weisheng Wang
The paper establishes identifiability results for constant‑width residual networks that use polynomial activations and bias terms. For activations of degree at least five with generic coefficients, the mapping from parameters to network outputs is generically one‑to‑one. For activations of degree at least three that lack a linear term after a suitable input translation, the generic fiber consists of translations of biases combined with scaling of weights and biases by real roots of unity, and these findings also apply to bias‑free networks. Additionally, the authors derive necessary conditions for critical points of the ResNet parametrization, which are important in singular learning theory.
By Isabel Dahlgren, Giovanni Luca Marchetti, Kathl\'en Kohn
arXiv:2609. 28500v1 Announce Type: cross Abstract: We study exact representability by bias-free shallow polynomial neural networks using algebraic geometry.
By Sepehr Akbari, Shahrzad Jamshidi
In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions. Indeed, while there has been a number of very recent results that establish ever-tighter lower bounds for the problem under linear and ReLU activation functions, less progress has been made towards the identification of novel polynomial-time tractable network architectures.
arXiv:2605. 09609v2 Announce Type: replace Abstract: We provide counterexamples to the unimodal minimal filling architecture conjecture for polynomial neural networks (PNNs) with power activation functions.
By Kevin Dao, Jose Israel Rodriguez
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
By Yulia Alexandr, Guido Mont\'ufar
arXiv:2610. 20069v1 Announce Type: new Abstract: What makes a deep network train, and what shape is its loss surface?
By Xiuyi Yang
Polynomial-Augmented Neural Networks (PANNs) merge deep neural networks with polynomial expansions to leverage the flexibility of DNNs and the rapid convergence of polynomials. The architecture introduces orthogonality constraints, basis pruning, and polynomial preconditioning to stabilize training and improve accuracy across diverse problems. Experiments show that PANNs outperform both pure DNNs and polynomial methods in approximating smooth and limited‑smoothness functions, as well as in solving partial differential equations.
By Madison Cooley, Shandian Zhe, Robert M. Kirby, Varun Shankar