arXiv AI June 29, 2026 By Thomas Chen

Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning

arXiv:2501. 07400v2 Announce Type: replace-cross Abstract: We derive explicit equations governing the cumulative biases and weights in Deep Learning with ReLU activation function, based on gradient descent for the Euclidean loss in the input layer, and under the assumption that the weights are, in a precise sense, adapted to the coordinate system distinguished by the activations.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

safety

arXiv Machine Learning

Jun 9

Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics

arXiv:2606. 09744v1 Announce Type: new Abstract: We study feed-forward ReLU networks with fixed readout and quadratic loss.

By Claudio Nordio

arXiv Machine Learning

Jul 7

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.

By Shuang Liang, Tom Jacobs, Guido Mont\'ufar

safety

arXiv Machine Learning

Jul 3

From Approximation to Emergence: A Theory of Deep Learning

arXiv:2607. 01311v1 Announce Type: new Abstract: Deep learning has outgrown any single mathematical explanation.

By Zhilin Zhao

llms safety

arXiv Machine Learning

Jun 9

Bulk-boundary decomposition of neural networks

arXiv:2511. 02003v2 Announce Type: replace Abstract: We present the bulk--boundary decomposition as a new framework for understanding the training dynamics of deep neural networks.

By Donghee Lee, Hye-Sung Lee, Jaeok Yi

arXiv Machine Learning

Jul 9

Network Dynamics-Based Framework for Understanding Deep Neural Networks

arXiv:2501. 02436v5 Announce Type: replace Abstract: Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning.

By Yuchen Lin, Yong Zhang, Sihan Feng, Hong Zhao

fine-tuning

arXiv Machine Learning

Jun 5

Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction

arXiv:2606. 05863v1 Announce Type: new Abstract: Grokking suggests that fitting the training data and learning a simple underlying rule may occur on different time scales.

By Hu Tan, Kuo Gai, Shihua Zhang

rag

Related stories

Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

From Approximation to Emergence: A Theory of Deep Learning

Bulk-boundary decomposition of neural networks

Network Dynamics-Based Framework for Understanding Deep Neural Networks

Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction