The paper introduces Neural Expansion, a geometric framework that explains how deep neural networks generalize by expanding directionally-selective ray coverage in input space. It defines Generalization Intervals (GIs) to measure stable prediction regions around training samples and shows through experiments that training progressively enlarges this coverage, especially along directions with low Jacobian singular values. This expansion accounts for correct test predictions, adversarial robustness, and handling of mislabeled or out-of-distribution data, offering a mechanistic complement to existing global theories.
arXiv:2503. 07325v2 Announce Type: replace Abstract: Understanding and certifying the behavior of modern deep neural networks remains a fundamental challenge in reliable machine learning.
arXiv:2606. 31282v1 Announce Type: new Abstract: Modern deep neural networks often contain far more parameters than needed to fit their training data, yet they achieve impressive generalization.
The paper introduces a pointwise generalization theory for fully connected deep neural networks, using a pointwise Riemannian Dimension derived from eigenvalues of learned feature representations across layers. This framework provides hypothesis-dependent, representation-aware generalization bounds that are significantly tighter than traditional size- or norm-based approaches, both theoretically and experimentally. The authors analytically identify structural properties that explain deep networks’ tractability and empirically show that the pointwise Riemannian Dimension captures feature compression, over‑parameterization effects, and optimizer bias.
arXiv:2606. 06772v1 Announce Type: cross Abstract: Understanding the generalization performance of over-parameterized neural networks has become a central topic in deep learning theory.
arXiv:2606. 26207v1 Announce Type: cross Abstract: Several theoretical works have tried to explain the adversarial vulnerability of deep neural networks through properties of high-dimensional geometry.
arXiv:2602.08302v3 Announce Type: replace-cross Abstract: Delayed generalization, where training accuracy becomes nearly perfect substantially before test accuracy, has often been studied in the cont...
arXiv:2510. 24616v4 Announce Type: replace-cross Abstract: For four decades statistical physics has been providing a framework to analyse neural networks.
arXiv:2504.16450v4 Announce Type: replace Abstract: We derive a differential equation that governs the evolution of the generalization gap when a model is trained by gradient descent-based methods. T...
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv:2410. 07719v4 Announce Type: replace Abstract: Despite being widely adopted as a canonical framework for learning robust models, adversarial training suffers from robust overfitting.
arXiv:2610.31783v1 Announce Type: new Abstract: We study the early training dynamics of nonlinear residual networks with blocks of zero hidden layers as a model of more general residual networks. We...
arXiv:2510.05606v2 Announce Type: replace Abstract: Fundamental limits to predictability are central to our understanding of many physical and computational systems. In deep learning, training outcom...