OpenAI Blog
Nonlinear computation in deep linear networks
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv Machine Learning
Jul 9
Network Dynamics-Based Framework for Understanding Deep Neural Networks
arXiv:2501. 02436v5 Announce Type: replace Abstract: Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning.
arXiv Machine Learning
Jun 9
Bulk-boundary decomposition of neural networks
arXiv:2511. 02003v2 Announce Type: replace Abstract: We present the bulk--boundary decomposition as a new framework for understanding the training dynamics of deep neural networks.
arXiv AI
Jun 29
Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning
arXiv:2501. 07400v2 Announce Type: replace-cross Abstract: We derive explicit equations governing the cumulative biases and weights in Deep Learning with ReLU activation function, based on gradient descent for the Euclidean loss in the input layer, and under the assumption that the weights are, in a precise sense, adapted to the coordinate system distinguished by the activations.
arXiv Machine Learning
Jun 10
Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss
arXiv:2402. 00152v5 Announce Type: replace Abstract: Constructing the architecture of a neural network is a challenging pursuit for the machine learning community, and the dilemma of whether to go deeper or wider remains a persistent question.
arXiv Machine Learning
Jun 24
A Theory of Saddle Escape in Deep Nonlinear Networks
arXiv:2605. 01288v3 Announce Type: replace Abstract: In deep networks with small initialization, training exhibits long plateaus separated by sharp feature-acquisition transitions.
arXiv Machine Learning
Jul 28
Variational Boosting for Physics-Informed Neural Networks
arXiv:2607. 23940v1 Announce Type: new Abstract: Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution.