The paper investigates when frequency decomposition aids Physics-Informed Neural Networks (PINNs) by introducing a dual‑branch, spectrally‑gated architecture (DBSG‑PINN) that separates low‑ and high‑frequency components. Experiments on five one‑dimensional PDE benchmarks show that frequency decomposition significantly reduces error—up to 59.2% on a multimodal wave problem—when the target solution is spectrally complex, but offers little improvement on smoother problems and can even worsen performance on a simple 1D wave benchmark. The adaptive gate’s effectiveness scales with the spectral richness of the solution, suggesting it exploits frequency structure rather than adding noise.
By Shubham Rai
arXiv:2606. 16575v1 Announce Type: new Abstract: Deep neural networks (DNNs) have achieved remarkable success in scientific computing, yet they often suffer from spectral bias in capturing oscillatory and multiscale behaviors.
By Yong Wang, Tao Zhou, Xuhui Meng
arXiv:2402.17595v2 Announce Type: replace-cross
Abstract: The phenomenon of implicit bias has attracted interest in recent years as a fundamental aspect of the remarkable generalizing ability of neur...
By Hong T. M. Chu, Xiaoyu Dong, Subhro Ghosh, Chi Thanh Lam, Soumendu Sundar Mukherjee
arXiv:2609.15355v1 Announce Type: cross
Abstract: We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by deep ReLU neural network...
By Shuhao Jiao
The paper analyzes gradient flow in a two‑layer linear neural network as the input, hidden, and output dimensions grow proportionally. Using a closed‑form solution combined with random matrix theory, it derives exact asymptotic spectral distributions for the weight matrices at all times and tracks how the learned subspace evolves. In the limit of small initialization, the study reveals a silent alignment phase where singular directions align quickly while singular values and loss stay unchanged, followed by a visible learning phase on a logarithmic timescale where singular values increase and loss decreases through a saddle‑to‑saddle dynamic that becomes continuous in the high‑dimensional limit.
By Simon Martin (DI-ENS, LPENS, SIERRA), Francis Bach (DI-ENS, SIERRA), Giulio Biroli (LPENS)
arXiv:2606. 28486v1 Announce Type: cross Abstract: The emergence of low-dimensional structures in the spectra of neural network weight matrices is a common empirical feature of trained models, but the dynamical origin of this phenomenon during learning remains an open problem.
By Chanju Park, Dario Bocchi, Francesco D'Amico, Biagio Lucini, Gert Aarts
Long-range learning is hard for recurrent networks trained with stochastic gradient descent, because the influence of a past input fades with the lag $\ell$, and if it fades too fast the dependence cannot be learned from finite data. This fade is captured by an envelope $f(\ell)$.
arXiv:2610.07419v1 Announce Type: new
Abstract: Implicit neural representations (INRs) are shaped by the spectral structure induced by their input encodings and activation functions. Existing methods...
By Tamir Shor, Or Litany, Alex Bronstein
arXiv:2609.15355v2 Announce Type: replace-cross
Abstract: We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by ReLU neural netw...
By Shuhao Jiao
arXiv:2509. 24882v2 Announce Type: replace Abstract: Neural scaling laws underlie many of the recent advances in deep learning, yet their theoretical understanding remains largely confined to linear models.
By Leonardo Defilippis, Yizhou Xu, Julius Girardin, Emanuele Troiani, Vittorio Erba, Lenka Zdeborov\'a, Bruno Loureiro, Florent Krzakala
arXiv:2606. 29519v1 Announce Type: new Abstract: Long-range learning is hard for recurrent networks trained with stochastic gradient descent, because the influence of a past input fades with the lag $\ell$, and if it fades too fast the dependence cannot be learned from finite data.
By Lorenzo Livi
The paper introduces Spectral-like Neural Discretisation (SpeND), a mesh‑free method that learns stencil weights via a neural network to approximate the modal response of a spectral operator across a specified band of wavenumbers. By projecting the network output onto the space of polynomial‑consistent weights, SpeND ensures exact consistency while minimizing dispersion and dissipation errors in a self‑supervised, physics‑agnostic manner. Experiments on disordered 2‑D node sets demonstrate that the learned fourth‑order operator matches the exact spectral response over a wider band than traditional LABFM or structured‑grid finite differences, and retains fourth‑order convergence upon refinement.
By Lucas Gerken Starepravo, Henry Broadley, Steven Lind, Jack R. C. King