arXiv Machine Learning
Jun 16
Finite-Width Neural Tangent Kernels from Feynman Diagrams
arXiv:2508. 11522v4 Announce Type: replace Abstract: Neural tangent kernels (NTKs) are a powerful tool for analyzing deep, non-linear neural networks.
Interpreting FCDNNs via RG on Exponential Family
arXiv:2606. 00157v1 Announce Type: cross Abstract: We consider establishing the interpretability theory of deep learning through constructing a corresponding relationship between the renormalization group (RG) method in statistical physics and the training process of deep neural networks (DNNs).
Related stories
arXiv Machine Learning
Jun 16
RepNet: Tackling spectral bias in deep neural networks via parameter reparameterization
arXiv:2606. 16575v1 Announce Type: new Abstract: Deep neural networks (DNNs) have achieved remarkable success in scientific computing, yet they often suffer from spectral bias in capturing oscillatory and multiscale behaviors.
arXiv Machine Learning
Jun 5
On Universality of Deep Equivariant Networks
arXiv:2510. 15814v2 Announce Type: replace-cross Abstract: Universality results for equivariant neural networks remain rare.
Hugging Face Trending Papers
Jun 22
Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv Machine Learning
Jun 19
Statistical Properties of Training & Generalization
arXiv:2606. 20299v1 Announce Type: cross Abstract: Deep learning has managed to evade numerous intuitions from classical statistics to achieve unprecedented performance on a number of real-world tasks.
arXiv Machine Learning
Jun 10
Rank Collapse, Fixed Points, and the Renormalization Group Structure of MLP Residual Networks
arXiv:2606. 10324v1 Announce Type: new Abstract: The analogy between deep neural network forward passes and renormalization group (RG) flows has been repeatedly noted in the literature, but existing treatments remain qualitative: depth is described as a coarse-graining scale, attention is likened to a partition function, and representations are said to flow toward fixed points.
arXiv Machine Learning
Jul 24
Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation
arXiv:2510. 24616v4 Announce Type: replace-cross Abstract: For four decades statistical physics has been providing a framework to analyse neural networks.
arXiv AI
Jun 29
Derivation of effective gradient flow equations and dynamical truncation of training data in Deep Learning
arXiv:2501. 07400v2 Announce Type: replace-cross Abstract: We derive explicit equations governing the cumulative biases and weights in Deep Learning with ReLU activation function, based on gradient descent for the Euclidean loss in the input layer, and under the assumption that the weights are, in a precise sense, adapted to the coordinate system distinguished by the activations.
arXiv Machine Learning
Jun 26
Theory of the Frequency Principle for General Deep Neural Networks
arXiv:1906. 09235v3 Announce Type: replace Abstract: Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Principle (F-Principle): a DNN tends to learn a target function from low to high frequencies during the training.
arXiv Machine Learning
Jun 16
Enhancing Physics-Informed Neural Networks Through Feature Engineering
arXiv:2502. 07209v4 Announce Type: replace Abstract: Physics-Informed Neural Networks (PINNs) seek to solve partial differential equations (PDEs) with deep learning.
arXiv AI
Jun 8
Generalization in Deep Neural Networks: Minimax Rates for Gradient Methods
arXiv:2606. 06772v1 Announce Type: cross Abstract: Understanding the generalization performance of over-parameterized neural networks has become a central topic in deep learning theory.
Hugging Face Trending Papers
Jun 9
Rank Collapse, Fixed Points, and the Renormalization Group Structure of MLP Residual Networks
The analogy between deep neural network forward passes and renormalization group (RG) flows has been repeatedly noted in the literature, but existing treatments remain qualitative: depth is described as a coarse-graining scale, attention is likened to a partition function, and representations are said to flow toward fixed points. No existing work has defined a measurable RG order parameter, tested it under controlled variation of the input distribution, or made quantitative predictions that are empirically verified.