The paper studies the symmetry properties of the differential neural tangent kernel (DNTK) for overparameterized physics‑informed neural networks. It shows that the infinite‑width DNTK is invariant under the orthogonal group for both two‑layer and deep networks, regardless of activation or bias. The authors further prove that integral operators induced by group‑orbit operators share spectra and that the fixed DNTK exhibits zonal symmetry on the unit sphere, enabling derivation of eigenvalue decay rates for shallow RePU networks with zero bias.
arXiv:2607. 10200v1 Announce Type: new Abstract: The Neural Tangent Kernel (NTK) is one powerful tool for analyzing the training dynamics of neural networks in the over-parameterized regime.
arXiv:2508. 11522v4 Announce Type: replace Abstract: Neural tangent kernels (NTKs) are a powerful tool for analyzing deep, non-linear neural networks.
arXiv:2607. 07845v1 Announce Type: new Abstract: The Hessian of the training loss governs the local geometry of the loss landscape, yet despite existing explanations for its largest eigenvalues, the origin of the vast multitude of vanishingly small eigenvalues remains elusive.
arXiv:2604. 14037v2 Announce Type: replace Abstract: Parameter space is not function space for neural network architectures.
arXiv:2610. 21964v1 Announce Type: cross Abstract: Neural networks often learn low-frequency components before high-frequency ones --- a phenomenon called spectral bias or termed F-principle.
We study the training dynamics of a feed-forward neural network (NN) through the analysis of the neural tangent kernel (NTK) in a phenomenologically relevant fit of parton distribution functions (PDFs...
arXiv:2608.24700v1 Announce Type: new Abstract: When a network has learned a function with a known symmetry, can that symmetry be moved through the parametrisation---is there a motion in parameter sp...
The paper investigates why Hessian spectra of trained deep learning models consistently show a bulk of eigenvalues near zero and a few isolated outliers. It proposes that these patterns arise from a hidden, highly symmetric reference configuration; when the model is perturbed (e.g., by architecture, data, or metric changes), the symmetry is broken, producing the observed cluster hierarchy. The authors analyze this mechanism in three‑layer ReLU networks and extend it to convolutional, graph, transformer models, the NTK, and related matrices such as layerwise Hessians and the Gauss‑Newton matrix.
arXiv:2511. 12398v2 Announce Type: replace Abstract: Deep neural networks have been widely used as universal approximators for functions with inherent physical structures, including permutation symmetry.
arXiv:2609.39020v1 Announce Type: new Abstract: We study how well deep neural networks approximate and learn spectral Barron functions. Recent studies have shown that these function classes can be ef...
The paper studies operator learning on function spaces using encoder–decoder architectures. It shows that as input and output resolutions grow, the induced kernels converge to a limiting kernel, enabling regularity assumptions independent of resolution. The authors derive upper and lower bounds for regularized stochastic gradient descent, extend the analysis to neural networks via the limiting neural tangent kernel, and provide error bounds and complexity guarantees for various kernel and encoding constructions.
arXiv:2606. 10913v1 Announce Type: new Abstract: We explore whether intrinsic symmetries of the training data lead to conserved quantities during gradient-flow training of neural networks.