arXiv Machine Learning
Jul 8
Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity
arXiv:2607. 05546v1 Announce Type: cross Abstract: We develop a unified function space theory of deep fully connected neural networks.
How Far Can Sharpness and Complexity Jointly Explain Generalization?
arXiv:2606. 29043v1 Announce Type: new Abstract: Sharpness and complexity are two central factors in the generalization analysis of deep neural networks.
Related stories
arXiv Machine Learning
Jun 2
Non-vacuous Generalization Bounds for Deep Neural Networks without any modification to the trained models
arXiv:2503. 07325v2 Announce Type: replace Abstract: Understanding and certifying the behavior of modern deep neural networks remains a fundamental challenge in reliable machine learning.
arXiv AI
Jun 9
Quantifying and Optimizing Simplicity via Polynomial Representations
arXiv:2605. 29823v2 Announce Type: replace Abstract: Deep networks often exhibit a preference for "simple" solutions, and such a simplicity bias is widely believed to play a key role in generalization.
arXiv AI
Jun 4
An Empirical Study of Data Scale, Model Complexity, and Input Modalities in Visual Generalization
arXiv:2606. 04409v1 Announce Type: cross Abstract: Modern deep neural networks usually have large parameter scales and nonlinear hierarchical structures, and they have achieved strong performance in computer vision.
arXiv AI
Jun 8
Generalization in Deep Neural Networks: Minimax Rates for Gradient Methods
arXiv:2606. 06772v1 Announce Type: cross Abstract: Understanding the generalization performance of over-parameterized neural networks has become a central topic in deep learning theory.
arXiv Machine Learning
Aug 4
Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View
arXiv:2608. 01357v1 Announce Type: new Abstract: Traditional approximation theory measures convergence rates in terms of the number of parameters or degrees of freedom.
Hugging Face Trending Papers
Jun 22
Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv Machine Learning
Jun 16
The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints
arXiv:2606. 16028v1 Announce Type: new Abstract: Modern deep learning architectures are increasingly multi-task and multi-modal, using a pretrained foundation model combined with task-specific, fine-tuned models.
arXiv Machine Learning
Jul 9
Adversarial Rademacher Complexity of Deep Neural Networks
arXiv:2211. 14966v2 Announce Type: replace Abstract: Deep neural networks (DNNs) are highly vulnerable to adversarial attacks.
arXiv Machine Learning
Jul 3
From Approximation to Emergence: A Theory of Deep Learning
arXiv:2607. 01311v1 Announce Type: new Abstract: Deep learning has outgrown any single mathematical explanation.
arXiv Machine Learning
Jun 2
Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structures
arXiv:2606. 00557v1 Announce Type: new Abstract: To understand how a neural network (NN) functions and makes predictions, it has become increasingly clear that analyzing only the input domain is insufficient -- one must also examine its internal inference mechanisms to capture the complete picture.
arXiv Machine Learning
Jul 8
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
arXiv:2607. 06151v1 Announce Type: new Abstract: Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting and reduced performance on unseen data.