OpenAI Blog
Nonlinear computation in deep linear networks
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv Machine Learning
Jul 9
Network Dynamics-Based Framework for Understanding Deep Neural Networks
arXiv:2501. 02436v5 Announce Type: replace Abstract: Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning.
arXiv Machine Learning
Jun 9
Bulk-boundary decomposition of neural networks
arXiv:2511. 02003v2 Announce Type: replace Abstract: We present the bulk--boundary decomposition as a new framework for understanding the training dynamics of deep neural networks.
arXiv Machine Learning
Aug 26
Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
arXiv:2602.08515v3 Announce Type: replace-cross Abstract: This work investigates shallow physics-informed neural networks (PINNs) for solving forward and inverse problems governed by nonlinear partia...
arXiv Machine Learning
1d ago
Circuit realization and hardware linearization of monotone operator equilibrium networks
The paper demonstrates that a resistor‑diode network’s port behavior solves a ReLU monotone operator equilibrium network, effectively realizing a neural network in analog hardware. It introduces hardware linearization to compute gradients directly in the circuit, enabling in‑hardware training demonstrated via device‑level simulation. The study also extends to cascaded networks for feedforward architectures and shows how different nonlinear elements yield distinct activation functions, including a novel diode ReLU from a non‑ideal diode model.
arXiv AI
Jun 29
Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning
arXiv:2501. 07400v2 Announce Type: replace-cross Abstract: We derive explicit equations governing the cumulative biases and weights in Deep Learning with ReLU activation function, based on gradient descent for the Euclidean loss in the input layer, and under the assumption that the weights are, in a precise sense, adapted to the coordinate system distinguished by the activations.
arXiv Machine Learning
Jun 10
Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss
arXiv:2402. 00152v5 Announce Type: replace Abstract: Constructing the architecture of a neural network is a challenging pursuit for the machine learning community, and the dilemma of whether to go deeper or wider remains a persistent question.