The paper examines why standard neural architectures struggle to generalize to longer inputs when solving dynamic programming (DP) problems. It shows that every finite min-plus DP can be represented as a shortest‑path problem on a directed acyclic graph, equivalently as a tropical polynomial whose extended Newton polyhedron captures the decision boundary of the winning path. The authors prove that the graph, polynomial, and polyhedron descriptions form isomorphic semirings at both the formal polynomial and computed function levels, and they demonstrate that the natural dimensionality‑reduction operations in this semiring are neither injective nor closed, revealing structural limitations that hinder length‑generalization.
By Richard F. M. Lim, Ruriko Yoshida
arXiv:2609. 28500v1 Announce Type: cross Abstract: We study exact representability by bias-free shallow polynomial neural networks using algebraic geometry.
By Sepehr Akbari, Shahrzad Jamshidi
The paper introduces a computational tropical geometry framework for symbolically analyzing neural networks with tropical activations. It presents an algorithm that computes the network’s linear regions as explicit unions of polyhedra, proves its correctness, and connects the number of linear regions to the monomials in the tropical expression. The authors also define the Hoffman constant to bound distances to the farthest linear region and release the open‑source Julia library TropicalNN.jl to implement these tools, demonstrating their use on proof‑of‑concept examples.
By Paul Lezeau, Thomas Walker, Yueqi Cao, Shiv Bhatia, Anthea Monod
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
By Yulia Alexandr, Guido Mont\'ufar
arXiv:2607. 11540v1 Announce Type: cross Abstract: We study tropical circuits with scalar multiplication gates, that is, algebraic circuits whose gates implement $\max$, $+$, or multiplication with a positive constant.
By Christoph Hertrich, Moritz Stargalla
arXiv:2410. 04907v2 Announce Type: replace-cross Abstract: In this paper we contribute to the frequently studied question of how to decompose a continuous piecewise linear (CPWL) function into a difference of two convex CPWL functions.
By Marie-Charlotte Brandenburg, Moritz Grillo, Christoph Hertrich
arXiv:2602. 06737v2 Announce Type: replace Abstract: We present a generalized framework for the range verification of neural networks featuring non-linear activation functions.
By Noah Schwartz, Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur, Tuhin Sahai, Susmit Jha
arXiv:2606. 26705v1 Announce Type: cross Abstract: Feedforward neural network (NN) expressivity is typically studied by emulating optimal basis-expansion schemes.
By Anastasis Kratsios, Simone Brugiapaglia, Bum Jun Kim, Gregory Cousins, Haitz S\'aez de Oc\'ariz Borde
arXiv:2609.25874v1 Announce Type: new
Abstract: Deep neural networks approximate functions by composing affine maps with nonlinear activations, but how composition itself creates approximation power...
By Wentao Huang, Haizhang Zhang
arXiv:2607. 21651v1 Announce Type: new Abstract: We prove that the maximum of $n$ real numbers is exactly representable by a ReLU network with two hidden layers for every $n\le 10$.
By Kilian Rue{\ss}, Gennadiy Averkov, Florestan Brunck, Moritz Grillo, Christoph Hertrich, Georg Loho, Jack Stade, Moritz Stargalla, Matthew Sun, Martin Winter
The paper presents a complete characterization of when two deep ReLU networks realize the same function, showing that this occurs iff one can be transformed into the other using a set of axioms from many‑valued logic. It introduces a symbolic calculus that maps networks to substitution graphs, proves a completeness theorem linking equivalent formulas, and provides an algorithm to reconstruct networks from these graphs. The framework yields a new compositional normal form for MV logic that preserves the algebraic structure of deep ReLU networks.
By Yani Zhang, Helmut B\"olcskei
The paper introduces Shallow Universal Polynomial Networks (SUPNs), a single‑hidden‑layer architecture that replaces standard activations with learnable multivariate polynomials, thereby increasing expressivity per unit. SUPNs achieve universal approximation by either widening the network or increasing polynomial complexity, and the authors prove convergence rates and quasi‑optimal parameter choices. Empirical tests across a wide range of smooth, nonsmooth, and discontinuous functions in up to ten dimensions show that SUPNs often outperform deep neural networks and Kolmogorov‑Arnold networks in accuracy and variability while requiring fewer parameters and lower training cost.
By Zachary Morrow, Michael Penwarden, Brian Chen, Aurya Javeed, Akil Narayan, John D. Jakeman