arXiv:2505. 21423v3 Announce Type: replace Abstract: The remarkable generalization properties of overparameterized networks are often attributed to implicit biases, such as norm minimization at small learning rates and low sharpness in the Edge-of-Stability regime.
arXiv:2607. 16720v1 Announce Type: new Abstract: Understanding deep neural networks remains a central challenge in machine learning.
The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.
arXiv:2611.06759v1 Announce Type: new Abstract: We study logistic regression for continual learning with linear predictors in a high-dimensional statistical setting. Existing theoretical work has lar...
arXiv:2603. 04895v2 Announce Type: replace-cross Abstract: Overparameterized ML models, including neural networks, typically induce underdetermined training objectives with multiple global minima.
The paper investigates why saliency maps of adversarially‑trained neural networks appear sparse. It provides a theoretical explanation for two‑layer ReLU networks, showing that as data size and neuron count grow, minimizers converge to a Bayes classifier with minimal gradient and Barron norm. The sparsity arises because α−−‑training with λ−1 attacks yields anisotropic gradient norms that favor axis‑aligned, sparse gradients, a claim supported by experimental evaluation of gradient α‑norms and sparsity thresholds.
arXiv:2502. 11152v4 Announce Type: replace-cross Abstract: The optimization foundations of deep linear networks have recently received significant attention.
The paper investigates whether flatter loss landscapes drive better generalization in the context of Grokking. It shows that sharpness‑aware minimization (SAM) alone does not reliably induce the transition to generalization, but when combined with weight decay, SAM can accelerate this transition up to four times faster. The authors provide a theoretical analysis using a minimal two‑layer ReLU model, demonstrating that flatness alone cannot distinguish memorizing from generalizing solutions, while weight decay favors generalization and a local stability window explains SAM’s effect.
We study logistic regression for continual learning with linear predictors in a high-dimensional statistical setting. Existing theoretical work has largely focused on continual linear regression with...
arXiv:2606. 06772v1 Announce Type: cross Abstract: Understanding the generalization performance of over-parameterized neural networks has become a central topic in deep learning theory.
arXiv:2606. 29951v1 Announce Type: new Abstract: Interpretable Mesomorphic Neural Networks (IMNs) offer a promising framework that combines the predictive power of deep neural networks with the interpretability of linear models.
arXiv:2607. 02671v1 Announce Type: cross Abstract: Benign overfitting and double descent have come to shape our understanding of generalization in deep learning, establishing that overfitting is not only compatible with good generalization but can actively benefit it.