Depth Separation for Neural Networks under Gaussian Measure
Read the original on arXiv Machine Learning →The paper investigates depth separation for neural networks in the context of $L^2$ approximation of radial functions under a standard Gaussian measure in $d$ dimensions. It demonstrates that a particular radial function can be approximated within a fixed small $L^2$ error by a depth‑3 ReLU network of polynomial size in $d$, yet any depth‑2 network approximating the same function with bounded weights must have size exponential in $d$. The authors prove a general result linking high‑frequency Hermite expansion mass to the limitations of depth‑2 networks with small width and bounded weights, and all results are formally verified in Lean 4.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.