arXiv:2607. 13042v1 Announce Type: new Abstract: This paper traces, with explicit numerical values, how PyTorch's automatic differentiation (AD) engine computes gradients for Physics-Informed Neural Network (PINN) training -- a setting that requires two levels of differentiation: computing the physics derivative $\hat{y}'(t)=d\hat{y}/dt$ through the network, and computing parameter gradients $\nabla_\theta L$ of a loss that itself depends on $\hat{y}'(t)$.
By Abdeladhim Tahimi
arXiv:2606. 16926v1 Announce Type: cross Abstract: Functional optimization problems are typically solved by optimizing the parameters of a fixed representation, such as a neural network, resulting in highly nonconvex losses that complicate both training and theoretical analysis.
By Daniel Csillag, Rodrigo Schuller, Pedro Dall'Antonia, Leonidas Guibas, Luiz Velho, Tiago Novello
arXiv:2609.15355v1 Announce Type: cross
Abstract: We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by deep ReLU neural network...
By Shuhao Jiao
arXiv:2604. 07328v3 Announce Type: replace Abstract: How does the choice of training data influence an AI model?
By Sam Gunn
arXiv:2609.15355v2 Announce Type: replace-cross
Abstract: We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by ReLU neural netw...
By Shuhao Jiao
We systematically investigate finite-difference (FD) derivative computation in Physics-Informed Neural Networks (PINNs) as an alternative to automatic differentiation (AD). On three benchmark PDEs we show that, with a properly calibrated step size, FD matches AD in accuracy on every problem while running faster across the full tested batch-size range and using substantially less GPU memory, and that a stochastic variant we propose outperforms AD on a stationary problem.
arXiv:2609.38786v1 Announce Type: new
Abstract: Differentiable learning typically assumes that the scalar objective evaluated in the forward pass and the gradient supplied to the optimizer in the bac...
By Ningkang Peng, Xiaoqian Peng, Yifan He, Anjie Hu, Chao Tan, Peirong Ma, Yanhui Gu
arXiv:2608. 11020v1 Announce Type: new Abstract: We systematically investigate finite-difference (FD) derivative computation in Physics-Informed Neural Networks (PINNs) as an alternative to automatic differentiation (AD).
By Maciej J. Mikulski, Tadeusz Uhl
arXiv:2607. 10589v1 Announce Type: cross Abstract: In contrast to most studies on neural network approximation theory that characterize results through a single parameter, such as the total number of network parameters, \cite{shen2020deep} pioneered the characterization of approximation rates as a joint function of the width parameter $N$ and the depth parameter $L$, thereby granting greater architectural flexibility.
By Yanming Lai, Defeng Sun, Yang Wang
arXiv:2607. 03148v1 Announce Type: cross Abstract: Activation functions are considered an essential primitive for neural nonlinearity, i.
By Muhammad Sabih, Frank Hannig, J\"urgen Teich
The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.
By Hao Yu
arXiv:2606. 26705v1 Announce Type: cross Abstract: Feedforward neural network (NN) expressivity is typically studied by emulating optimal basis-expansion schemes.
By Anastasis Kratsios, Simone Brugiapaglia, Bum Jun Kim, Gregory Cousins, Haitz S\'aez de Oc\'ariz Borde