OpenAI Blog
Learning sparse neural networks through L₀ regularization
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv Machine Learning
Aug 11
Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks
arXiv:2502. 11152v4 Announce Type: replace-cross Abstract: The optimization foundations of deep linear networks have recently received significant attention.
arXiv Machine Learning
6d ago
Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness
arXiv:2608. 11479v1 Announce Type: new Abstract: We establish convergence guarantees of gradient descent for general feedforward neural networks of arbitrary width or depth, with no special requirements on the initialization or dataset.
arXiv Machine Learning
Jun 10
Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss
arXiv:2402. 00152v5 Announce Type: replace Abstract: Constructing the architecture of a neural network is a challenging pursuit for the machine learning community, and the dilemma of whether to go deeper or wider remains a persistent question.
arXiv Machine Learning
Jun 8
Conflicting Biases at the Edge of Stability: Norm versus Sharpness Regularization
arXiv:2505. 21423v3 Announce Type: replace Abstract: The remarkable generalization properties of overparameterized networks are often attributed to implicit biases, such as norm minimization at small learning rates and low sharpness in the Edge-of-Stability regime.
arXiv Machine Learning
Jun 30
Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization
arXiv:2505. 22578v2 Announce Type: replace Abstract: The optimization of neural networks under weight decay remains poorly understood from a theoretical standpoint.
arXiv AI
Jul 15
Mathematics of Data Science
arXiv:2607. 11938v1 Announce Type: cross Abstract: This book is about the mathematical foundations of data science.