arXiv Machine Learning
Nearly Tight Rademacher Bounds for Sparsely Activated Neural Networks
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
Sep 8
Nearly Tight Rademacher Bounds for Sparsely Activated Neural Networks
An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input-dependent sparsity in the one-hidden-layer ReLU model of Awasthi et al.
arXiv Machine Learning
Oct 5
Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds
arXiv:2610.02253v1 Announce Type: new Abstract: The universal approximation property of dropout neural networks does not by itself describe the network size required for an accurate random realizatio...
arXiv Computation and Language
Sep 15
Query-Oblivious Coresets for Softmax Attention: Improved Bounds and Efficient Constructions
arXiv:2609.06327v2 Announce Type: replace-cross Abstract: A query-oblivious coreset for a softmax-attention head is a subset of the key-value pairs whose attention output is within $\varepsilon$ of t...
arXiv Machine Learning
Sep 11
Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU$^k$ Networks on the Sphere
arXiv:2510. 04060v3 Announce Type: replace-cross Abstract: We establish two related but logically distinct results for shallow ReLU$^k$ neural networks on the unit sphere $\SS^d$.
arXiv Machine Learning
Jul 10
A law of robustness for two-layer neural networks with arbitrary weights
arXiv:2607. 07778v1 Announce Type: new Abstract: Bubeck, Li and Nagaraj conjectured that, for generic data, any two-layer neural network with $m$ neurons that fits $n$ noisy labels must have Lipschitz constant at least of order $\sqrt{n/m}$, with no restriction on the size of the weights.
arXiv Machine Learning
Sep 11
Near-optimal estimates for the $\ell^p$-Lipschitz constants of deep random ReLU neural networks
This paper investigates the ρ^p-Lipschitz constants of deep ReLU neural networks with random weights drawn from a He‑style initialization. For zero‑bias networks, it provides high‑probability upper and lower bounds that differ by at most a logarithmic factor in depth, and shows a sharp contrast between the regimes p∈[1,2) and p∈[2,∞], with the former behaving like the Euclidean norm of a Gaussian vector and the latter like its dual norm. The analysis is extended to networks with non‑zero biases from symmetric distributions, yielding bounds that differ by a logarithmic factor in width and a linear factor in depth.