arXiv AI
Sep 2
The Alexander-Hirschowitz theorem for neurovarieties
arXiv:2511. 19703v2 Announce Type: replace-cross Abstract: We study the dimension and identifiability of neurovarieties associated to polynomial neural networks.
Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networks
The paper proposes a conjecture that composing a fixed number of distinct nonconstant polynomials with a generic high‑degree polynomial produces linearly independent polynomials, extending Newman–Slater’s theorem. The authors prove the conjecture for two polynomials and for any number when the degrees are bounded, and they show how these results explain the parameter symmetries of deep fully connected neural networks with generic polynomial activations. In particular, for architectures with layer‑specific activations of increasing degree, the conjecture’s proven cases fully characterize the parameter sets that yield the same end‑to‑end network function, and it also resolves the identifiability of shallow polynomial networks.
Related stories
arXiv Machine Learning
Jul 24
New Complexity-Theoretic Frontiers of Tractability for Neural Network Training
arXiv:2607. 20811v1 Announce Type: new Abstract: In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions.
arXiv Machine Learning
Jun 30
Singular Learning and Occam's Razor in Deep Monomial Networks
arXiv:2606. 28464v1 Announce Type: new Abstract: In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture.
Hugging Face Trending Papers
Jul 23
New Complexity-Theoretic Frontiers of Tractability for Neural Network Training
In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions. Indeed, while there has been a number of very recent results that establish ever-tighter lower bounds for the problem under linear and ReLU activation functions, less progress has been made towards the identification of novel polynomial-time tractable network architectures.
arXiv Machine Learning
Jun 19
Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects
arXiv:2605. 09609v2 Announce Type: replace Abstract: We provide counterexamples to the unimodal minimal filling architecture conjecture for polynomial neural networks (PNNs) with power activation functions.
arXiv Machine Learning
Jun 16
Constraining the outputs of ReLU neural networks
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
arXiv Machine Learning
Aug 26
Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
Polynomial-Augmented Neural Networks (PANNs) merge deep neural networks with polynomial expansions to leverage the flexibility of DNNs and the rapid convergence of polynomials. The architecture introduces orthogonality constraints, basis pruning, and polynomial preconditioning to stabilize training and improve accuracy across diverse problems. Experiments show that PANNs outperform both pure DNNs and polynomial methods in approximating smooth and limited‑smoothness functions, as well as in solving partial differential equations.
arXiv Machine Learning
Jun 15
On the Geometry and Optimization of Polynomial Convolutional Networks
arXiv:2410. 00722v3 Announce Type: replace Abstract: We study convolutional neural networks with monomial activation functions.
arXiv Machine Learning
Jun 4
Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability
arXiv:2606. 04754v1 Announce Type: new Abstract: Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged.
arXiv Machine Learning
Jul 8
Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity
arXiv:2607. 05546v1 Announce Type: cross Abstract: We develop a unified function space theory of deep fully connected neural networks.
Hugging Face Trending Papers
Jul 21
Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization
The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error. However, the uniqueness of such neural network representations is not guaranteed, raising questions about practical identifiability.
arXiv Machine Learning
Aug 4
Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View
arXiv:2608. 01357v1 Announce Type: new Abstract: Traditional approximation theory measures convergence rates in terms of the number of parameters or degrees of freedom.