The paper establishes identifiability results for constant‑width residual networks that use polynomial activations and bias terms. For activations of degree at least five with generic coefficients, the mapping from parameters to network outputs is generically one‑to‑one. For activations of degree at least three that lack a linear term after a suitable input translation, the generic fiber consists of translations of biases combined with scaling of weights and biases by real roots of unity, and these findings also apply to bias‑free networks. Additionally, the authors derive necessary conditions for critical points of the ResNet parametrization, which are important in singular learning theory.
arXiv:2606. 28464v1 Announce Type: new Abstract: In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture.
The paper proposes a conjecture that composing a fixed number of distinct nonconstant polynomials with a generic high‑degree polynomial produces linearly independent polynomials, extending Newman–Slater’s theorem. The authors prove the conjecture for two polynomials and for any number when the degrees are bounded, and they show how these results explain the parameter symmetries of deep fully connected neural networks with generic polynomial activations. In particular, for architectures with layer‑specific activations of increasing degree, the conjecture’s proven cases fully characterize the parameter sets that yield the same end‑to‑end network function, and it also resolves the identifiability of shallow polynomial networks.
arXiv:2410. 00722v3 Announce Type: replace Abstract: We study convolutional neural networks with monomial activation functions.
The paper proves the weak composition‑independence conjectures of Kohn, Marchetti, Massarenti and Mella by employing deformations of rational curves and a single kernel of the linearized equation, with Riemann‑Roch providing a contradiction from tangent fields with controlled poles. This result establishes that a neural network using a general polynomial activation function of sufficiently large degree is identifiable.
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
arXiv:2603. 12785v2 Announce Type: replace Abstract: Three-layer neural networks are known to form singular learning models, and their Bayesian asymptotic behavior is governed by the learning coefficient, or real log canonical threshold.
arXiv:2511. 19703v2 Announce Type: replace-cross Abstract: We study the dimension and identifiability of neurovarieties associated to polynomial neural networks.
arXiv:2607. 04597v1 Announce Type: new Abstract: In this paper, we study the universal approximation property of residual neural networks, and obtain some new results.
arXiv:2607. 23397v1 Announce Type: new Abstract: Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood.
arXiv:2607. 20811v1 Announce Type: new Abstract: In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions.
arXiv:2609. 03626v1 Announce Type: cross Abstract: Rigorous results show that feedforward neural networks can overcome the curse of dimensionality in the numerical approximation of high-dimensional partial differential equations (PDEs), but comparatively little is known about residual neural networks (ResNets) in the nonlinear PDE setting.
arXiv:2610. 20069v1 Announce Type: new Abstract: What makes a deep network train, and what shape is its loss surface?