arXiv Machine Learning
Jul 8
Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity
arXiv:2607. 05546v1 Announce Type: cross Abstract: We develop a unified function space theory of deep fully connected neural networks.
The Geometry of Limits in ReLU Function Spaces
The paper investigates the geometry and closure properties of function spaces realized by ReLU neural networks with fixed architecture. It proves that the minimal number of convex pieces in continuous piecewise-linear functions is lower semicontinuous under pointwise convergence, and uses this to show that shallow scalar ReLU realization spaces are closed in the weighted supremum norm, while providing a counterexample that demonstrates non-closedness for a specific architecture. The work also characterizes the role of a blow‑up residue in non‑closedness and offers new geometric and functional‑analytic tools for analyzing ReLU realization spaces.
Related stories
arXiv Machine Learning
Aug 5
Neural network realization of binary refinement iterates via a two-chart atlas selector
arXiv:2608. 02624v1 Announce Type: cross Abstract: Refinement operators generate many functions used in wavelet constructions, subdivision schemes, and geometric modeling.
arXiv Machine Learning
Oct 9
NP-Hardness of Minimizing Neurons in Two-Hidden-Layer ReLU Neural Networks
arXiv:2610. 11313v1 Announce Type: new Abstract: A fundamental question in neural network architecture optimization is whether the minimum hidden-neuron count required to approximate a target function within a prescribed tolerance can be computed efficiently.
arXiv Machine Learning
Jun 9
Characterizing the Discrete Geometry of ReLU Networks
arXiv:2606. 07728v1 Announce Type: new Abstract: It is well established that ReLU networks define continuous piecewise-linear functions, and that their linear regions are polyhedra in the input space.
arXiv Machine Learning
Jul 27
Shallower ReLU Network Representations via Exact Linear Algebra
arXiv:2607. 21651v1 Announce Type: new Abstract: We prove that the maximum of $n$ real numbers is exactly representable by a ReLU network with two hidden layers for every $n\le 10$.
arXiv Machine Learning
Sep 15
ReLU Neural Network Approximation to Smooth Functional Operator: Dimensional Decay and Error Analysis
arXiv:2609.15355v1 Announce Type: cross Abstract: We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by deep ReLU neural network...
arXiv Machine Learning
Jun 16
Constraining the outputs of ReLU neural networks
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
arXiv Machine Learning
Aug 27
Representing MAX functions using two-hidden-layer ReLU networks
arXiv:2608. 25221v1 Announce Type: new Abstract: We study exact representations of $\mathrm{MAX}_N(x)=\max{x_1,\ldots,x_N}$ using two-hidden-layer ReLU neural networks.
arXiv Machine Learning
Jul 24
New Complexity-Theoretic Frontiers of Tractability for Neural Network Training
arXiv:2607. 20811v1 Announce Type: new Abstract: In spite of the fundamental role of neural networks in contemporary machine learning research, our understanding of the computational complexity of optimally training neural networks remains incomplete even when dealing with the simplest kinds of activation functions.
arXiv Machine Learning
Jun 2
Multigrade Neural Network Approximation
arXiv:2601. 16884v3 Announce Type: replace Abstract: We study multigrade deep learning (MGDL) as a principled framework for structured error refinement in deep neural networks.
arXiv Machine Learning
Jul 14
Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width
arXiv:2607. 10589v1 Announce Type: cross Abstract: In contrast to most studies on neural network approximation theory that characterize results through a single parameter, such as the total number of network parameters, \cite{shen2020deep} pioneered the characterization of approximation rates as a joint function of the width parameter $N$ and the depth parameter $L$, thereby granting greater architectural flexibility.
arXiv Machine Learning
Aug 26
Parameterized Complexity of $L_p$-Lipschitz Constants for Input Convex Neural Networks and $L_p$-Norm Maximization over Zonotopes
arXiv:2608.24865v1 Announce Type: cross Abstract: Lipschitz constants are a standard way to quantify the sensitivity of neural networks to small input perturbations, but computing them is difficult e...