arXiv Machine Learning
Jun 15
On the Geometry and Optimization of Polynomial Convolutional Networks
arXiv:2410. 00722v3 Announce Type: replace Abstract: We study convolutional neural networks with monomial activation functions.
Tubular Neighbourhoods of Pfaffian Sets and Applications to Neural Networks
arXiv:2607. 08370v1 Announce Type: cross Abstract: We derive bounds for the volume of tubular neighbourhoods of smooth Pfaffian hypersurfaces, generalising known results for algebraic varieties.
Related stories
arXiv Machine Learning
Jun 19
Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects
arXiv:2605. 09609v2 Announce Type: replace Abstract: We provide counterexamples to the unimodal minimal filling architecture conjecture for polynomial neural networks (PNNs) with power activation functions.
arXiv Machine Learning
Jun 10
Upper Bounds for Local Learning Coefficients of Three-Layer Neural Networks
arXiv:2603. 12785v2 Announce Type: replace Abstract: Three-layer neural networks are known to form singular learning models, and their Bayesian asymptotic behavior is governed by the learning coefficient, or real log canonical threshold.
arXiv Machine Learning
Jun 16
Constraining the outputs of ReLU neural networks
arXiv:2508. 03867v2 Announce Type: replace-cross Abstract: We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space.
arXiv Machine Learning
Jun 30
Singular Learning and Occam's Razor in Deep Monomial Networks
arXiv:2606. 28464v1 Announce Type: new Abstract: In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture.
arXiv Machine Learning
Jul 28
A Statistical Difference between Single-Layer Learning and Hierarchical Learning in Wide Neural Networks
arXiv:2607. 23397v1 Announce Type: new Abstract: Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood.
arXiv Machine Learning
Jul 7
Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One
arXiv:2607. 04597v1 Announce Type: new Abstract: In this paper, we study the universal approximation property of residual neural networks, and obtain some new results.
arXiv Machine Learning
Jun 25
Rational Neural Networks have Expressivity Advantages
arXiv:2602. 12390v2 Announce Type: replace Abstract: We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such as ELU, LeakyReLU, LogSigmoid, PReLU, ReLU, SELU, CELU, Sigmoid, SiLU, Mish, Softplus, Tanh, Softmin, Softmax, and LogSoftmax.
arXiv Machine Learning
Jul 9
On Explicit Super-Expressive Approximation for Neural Networks
arXiv:2607. 06781v1 Announce Type: new Abstract: In this work, we investigate the fixed-architecture neural network approximation with explicit parameter bounds and elementary activations.
arXiv Machine Learning
Jul 3
Fast approximation and learning of binary classification tasks in o-minimal structures using ReLU neural networks
arXiv:2607. 01266v1 Announce Type: cross Abstract: We study binary classification problems whose decision sets are given by definable sets in o-minimal expansions of the real field.
Hugging Face Trending Papers
Jul 7
On Explicit Super-Expressive Approximation for Neural Networks
In this work, we investigate the fixed-architecture neural network approximation with explicit parameter bounds and elementary activations. While prior work demonstrated super-expressive approximation using fixed-size networks, they lack quantitative and non-asymptotic characterizations of parameter magnitude with respect to the approximation error.
arXiv Machine Learning
Jul 10
A law of robustness for two-layer neural networks with arbitrary weights
arXiv:2607. 07778v1 Announce Type: new Abstract: Bubeck, Li and Nagaraj conjectured that, for generic data, any two-layer neural network with $m$ neurons that fits $n$ noisy labels must have Lipschitz constant at least of order $\sqrt{n/m}$, with no restriction on the size of the weights.