← Back to all news
arXiv Machine Learning By Ruijia Zhang, Soufiane Hayou

Width and Depth Limits Commute During Training

Read the original on arXiv Machine Learning →

arXiv:2610. 30729v1 Announce Type: new Abstract: Recent work established that the infinite-width and infinite-depth limits of residual networks commute at initialization when residual branches are scaled by $1/\sqrt{depth}$.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 24

Layer-wise Geometric Approximation Rates for Deep Networks

arXiv:2604. 20219v2 Announce Type: replace Abstract: Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for the final output and leaves the role of intermediate layers largely unclear.

By Shijun Zhang, Zuowei Shen, Yuesheng Xu
More like this →
arXiv Machine Learning
Oct 23

The Mathematical Theory of Double Descent as Shrinkage, not Overfitting: Matching Bounds for Path-Regularized Multilayer Neural Networks

The paper presents a nonasymptotic generalization theory for multilayer fully connected neural networks trained with path regularization, covering both underparameterized and overparameterized regimes without assuming bounded loss, sparsity, or width/depth limits. It explicitly models approximation error, resolves an open problem on approximation rates in generalized Barron spaces, and establishes matching upper and lower bounds that show the error curve is non‑monotone—descending, rising, then descending again—as width increases. The analysis reveals that the rise is due to shrinkage imposed by regularization rather than overfitting, and this effect cannot be eliminated by width‑independent regularization.

By Hao Yu
More like this →
arXiv Statistics ML
5d ago

Early learning in large residual networks

arXiv:2610.31783v1 Announce Type: new Abstract: We study the early training dynamics of nonlinear residual networks with blocks of zero hidden layers as a model of more general residual networks. We...

By Vladimir Narovlansky, Boris Hanin
More like this →