arXiv:2609. 28500v1 Announce Type: cross Abstract: We study exact representability by bias-free shallow polynomial neural networks using algebraic geometry.
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
The paper proposes a conjecture that composing a fixed number of distinct nonconstant polynomials with a generic high‑degree polynomial produces linearly independent polynomials, extending Newman–Slater’s theorem. The authors prove the conjecture for two polynomials and for any number when the degrees are bounded, and they show how these results explain the parameter symmetries of deep fully connected neural networks with generic polynomial activations. In particular, for architectures with layer‑specific activations of increasing degree, the conjecture’s proven cases fully characterize the parameter sets that yield the same end‑to‑end network function, and it also resolves the identifiability of shallow polynomial networks.
The paper investigates the polynomial coefficients of lightning self‑attention, treating them as coordinates of an algebraic variety. In the single‑token case it identifies the coefficient variety as a rank‑constrained Chow‑type variety and derives algebraic equations; for multiple tokens it shows that linear relations reduce the geometry to coefficients involving interactions between distinct tokens, characterized by a common linear factor and a low‑rank condition. The authors provide explicit families of determinantal, Veronese‑type, and Sylvester resultant‑based invariants, and in the rank‑one case give pencil and flattening equations that define the variety set‑theoretically, with small‑dimension computations confirming the theoretical generators.
arXiv:2609.25776v1 Announce Type: cross Abstract: Nonlinear activations can create equivariant interactions between irreducible representations that linear maps cannot. We use the Gaussian degree dec...
arXiv:2511. 19703v2 Announce Type: replace-cross Abstract: We study the dimension and identifiability of neurovarieties associated to polynomial neural networks.
arXiv:2605. 09609v2 Announce Type: replace Abstract: We provide counterexamples to the unimodal minimal filling architecture conjecture for polynomial neural networks (PNNs) with power activation functions.
arXiv:2410. 00722v3 Announce Type: replace Abstract: We study convolutional neural networks with monomial activation functions.
The paper introduces a new mathematical framework for polynomial group convolutional neural networks (PGCNNs) using graded group algebras. It presents two natural parametrizations of the architecture—based on Hadamard and Kronecker products—that are related by a linear map. The authors compute the dimension of the resulting neuromanifold, show it depends only on the number of layers and group size, and describe the general fiber of the Kronecker parametrization, conjecturing a similar description for the Hadamard case, supported by explicit computations for small groups and shallow networks.
arXiv:2607. 13749v1 Announce Type: new Abstract: Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately.
Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately. What happens at the extreme of this spectrum, when the architecture's expressible function class collapses to a finite-dimensional algebraic variety?
arXiv:2606. 28464v1 Announce Type: new Abstract: In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture.
The paper examines why standard neural architectures struggle to generalize to longer inputs when solving dynamic programming (DP) problems. It shows that every finite min-plus DP can be represented as a shortest‑path problem on a directed acyclic graph, equivalently as a tropical polynomial whose extended Newton polyhedron captures the decision boundary of the winning path. The authors prove that the graph, polynomial, and polyhedron descriptions form isomorphic semirings at both the formal polynomial and computed function levels, and they demonstrate that the natural dimensionality‑reduction operations in this semiring are neither injective nor closed, revealing structural limitations that hinder length‑generalization.