The paper examines why standard neural architectures struggle to generalize to longer inputs when solving dynamic programming (DP) problems. It shows that every finite min-plus DP can be represented as a shortest‑path problem on a directed acyclic graph, equivalently as a tropical polynomial whose extended Newton polyhedron captures the decision boundary of the winning path. The authors prove that the graph, polynomial, and polyhedron descriptions form isomorphic semirings at both the formal polynomial and computed function levels, and they demonstrate that the natural dimensionality‑reduction operations in this semiring are neither injective nor closed, revealing structural limitations that hinder length‑generalization.
arXiv:2607. 19573v1 Announce Type: cross Abstract: Structural generalization has been measured repeatedly by several benchmarks, yet it has never been formally defined.
The paper introduces a computational tropical geometry framework for symbolically analyzing neural networks with tropical activations. It presents an algorithm that computes the network’s linear regions as explicit unions of polyhedra, proves its correctness, and connects the number of linear regions to the monomials in the tropical expression. The authors also define the Hoffman constant to bound distances to the farthest linear region and release the open‑source Julia library TropicalNN.jl to implement these tools, demonstrating their use on proof‑of‑concept examples.
arXiv:2609. 28500v1 Announce Type: cross Abstract: We study exact representability by bias-free shallow polynomial neural networks using algebraic geometry.
arXiv:2608. 13433v1 Announce Type: cross Abstract: Transformer-based language models are known to sometimes generalize to sequences longer than seen during training, but we lack a precise characterization of which tasks admit length generalization.
arXiv:2605. 20919v3 Announce Type: replace-cross Abstract: Sutra is a typed, purely functional programming language whose compiled forward pass is a PyTorch neural network.
Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately. What happens at the extreme of this spectrum, when the architecture's expressible function class collapses to a finite-dimensional algebraic variety?
arXiv:2602.13106v2 Announce Type: replace-cross Abstract: In recent years, there has been growing interest in understanding neural architectures' ability to learn to execute discrete algorithms, a li...
arXiv:2608. 08118v1 Announce Type: new Abstract: There are several methods for searching for graphs with prescribed properties, such as SAT solvers and specialized generators.
arXiv:2511. 14953v2 Announce Type: replace-cross Abstract: Discrete structures are currently second-class in differentiable programming.
arXiv:2603. 25414v4 Announce Type: replace-cross Abstract: A prevailing assumption in machine learning is that model correctness must be enforced after the fact.
arXiv:2606. 10806v1 Announce Type: new Abstract: Moonshine is an autonomous agent whose central objective is to generate mathematical conjectures.
arXiv:2404.11624v3 Announce Type: replace-cross Abstract: We introduce Token Space, a categorical framework for AI computations based on explicit structural records. Five theses guide it: object inte...