← Back to all news
arXiv Machine Learning

High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercube

The paper establishes high‑probability bounds on mixed input derivatives for wide random neural networks whose activation derivatives grow factorially, with a focus on anh networks initialized with Xavier weights. For scalar‑output anh networks with Gaussian weights, the authors prove that when the hidden width exceeds a depth‑dependent threshold, the derivative of any order satisfies a bound that is independent of depth for first‑order derivatives and grows at most polynomially with depth for higher‑order mixed derivatives. These results yield high‑probability estimates for the Euclidean Lipschitz constant and weighted Sobolev norms, linking the regularity of network realizations to quasi‑Monte Carlo integration and its potential impact on QMC‑based training.

Related stories

arXiv Machine Learning
Sep 11

Near-optimal estimates for the $\ell^p$-Lipschitz constants of deep random ReLU neural networks

This paper investigates the ρ^p-Lipschitz constants of deep ReLU neural networks with random weights drawn from a He‑style initialization. For zero‑bias networks, it provides high‑probability upper and lower bounds that differ by at most a logarithmic factor in depth, and shows a sharp contrast between the regimes p∈[1,2) and p∈[2,∞], with the former behaving like the Euclidean norm of a Gaussian vector and the latter like its dual norm. The analysis is extended to networks with non‑zero biases from symmetric distributions, yielding bounds that differ by a logarithmic factor in width and a linear factor in depth.

By Sjoerd Dirksen, Patrick Finke, Paul Geuchen, Dominik St\"oger, Felix Voigtlaender
safety
More like this →
arXiv Machine Learning
Jul 22

Boundary-Adapted PINNs for Elliptic Dirichlet Problems: $H^2(\Omega)$ A Priori Error Bounds with Application to Mean Escape Time Computation

arXiv:2607. 19167v1 Announce Type: cross Abstract: Motivated by the numerical computation of the Mean Escape Time (MET) $\tau:\Omega\to\mathbb{R}$ of a stochastic process from a bounded domain $\Omega\subseteq\mathbb{R}^d$, we study elliptic Dirichlet boundary value problems (BVPs) using boundary-enforced Physics-Informed Neural Networks (PINNs), in which the Dirichlet condition is imposed exactly by multiplying the network output with a predefined distance-to-boundary approximation $\rho$.

By Nathanael Tepakbong, Jun Fan, Xiang Zhou, Ding-Xuan Zhou
More like this →
arXiv Machine Learning
Sep 4

Residual neural networks overcome the curse of dimensionality for semilinear heat equations

arXiv:2609. 03626v1 Announce Type: cross Abstract: Rigorous results show that feedforward neural networks can overcome the curse of dimensionality in the numerical approximation of high-dimensional partial differential equations (PDEs), but comparatively little is known about residual neural networks (ResNets) in the nonlinear PDE setting.

By Ilkhom Mukhammadiev, Diyora Salimova
More like this →