arXiv:2606. 07728v1 Announce Type: new Abstract: It is well established that ReLU networks define continuous piecewise-linear functions, and that their linear regions are polyhedra in the input space.
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
arXiv:2607. 21651v1 Announce Type: new Abstract: We prove that the maximum of $n$ real numbers is exactly representable by a ReLU network with two hidden layers for every $n\le 10$.
The paper introduces a general theoretical framework for fibrations on graphs labeled by a commutative monoid, extending the classic theory of graph fibrations to weighted and algebraically labeled graphs. It also accommodates approximate fibrations and demonstrates how this framework can be used to compress arbitrary neural networks, including CNNs, providing a solid theoretical basis for recent findings on fibration symmetries in geometric deep learning.
arXiv:2608.23877v1 Announce Type: new Abstract: We prove a depth hierarchy for ReLU neural networks in which every additional ReLU layer can save exponentially many neurons. For every $\ell\geq 3$, a...
In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions. Indeed, while there has been a number of very recent results that establish ever-tighter lower bounds for the problem under linear and ReLU activation functions, less progress has been made towards the identification of novel polynomial-time tractable network architectures.
arXiv:2607. 20811v1 Announce Type: new Abstract: In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions.
The paper applies parameterised graph theory to tensor networks, showing that cutwidth and tree‑cutwidth bound the bond‑dimension overhead needed to represent a tensor‑network state as a matrix product state or tree tensor network. It derives graph‑dependent upper bounds on the sample and computational complexity of tensor‑network tomography, introducing a new graph parameter called learning complexity. Finally, it extends the framework to an agnostic learner that approximates any state with a tensor‑network state of given bond dimension, providing explicit graph‑dependent complexity bounds.
arXiv:2607. 10589v1 Announce Type: cross Abstract: In contrast to most studies on neural network approximation theory that characterize results through a single parameter, such as the total number of network parameters, \cite{shen2020deep} pioneered the characterization of approximation rates as a joint function of the width parameter $N$ and the depth parameter $L$, thereby granting greater architectural flexibility.
arXiv:2607. 07035v1 Announce Type: cross Abstract: The architecture of deep feedforward neural networks is ubiquitous in deep learning, either as a whole system or as a subnetwork of other architectures, and thus its mechanism is a key ingredient of the black box of neural networks.
arXiv:2606. 31856v1 Announce Type: new Abstract: We study layered models, including feedforward networks, ResNets, and transformers, by limiting each layer to a width of $d = 3$, i.
arXiv:2608. 02624v1 Announce Type: cross Abstract: Refinement operators generate many functions used in wavelet constructions, subdivision schemes, and geometric modeling.
arXiv:2605. 00725v2 Announce Type: replace Abstract: Topological neural networks have emerged as effective tools for modeling higher-order relational structures beyond pairwise graphs, including hypergraphs, simplicial complexes, and cell complexes.