arXiv:2607. 02003v1 Announce Type: cross Abstract: Although neural networks are remarkably effective, their underlying optimization principles remain theoretically elusive, often characterized by non-convex landscapes and stochastic heuristics.
arXiv:2311. 15365v3 Announce Type: replace Abstract: We study an idealized training process for deep neural networks in a continuous-depth, mean-field model in which each layer is parameterized by a probability measure on a Euclidean parameter space.
arXiv:2505. 12430v2 Announce Type: replace Abstract: Recently, innovative adaptations of the Ritz method incorporating deep learning have been developed, known as the Deep Ritz Method.
arXiv:2608. 09523v1 Announce Type: new Abstract: Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success.
arXiv:2606. 14954v1 Announce Type: cross Abstract: We develop a general framework for analyzing representation costs of parametric data-fitting methods through their parameter-space regularizers.
The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.
arXiv:2501.14993v4 Announce Type: replace-cross Abstract: The proximal algorithm is a powerful tool to minimize nonlinear and nonsmooth functionals in a general metric space. Motivated by the recent...
arXiv:2402. 00152v5 Announce Type: replace Abstract: Constructing the architecture of a neural network is a challenging pursuit for the machine learning community, and the dilemma of whether to go deeper or wider remains a persistent question.
arXiv:2609. 19937v1 Announce Type: cross Abstract: Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights.
arXiv:2607. 15702v2 Announce Type: replace-cross Abstract: We develop a non-asymptotic approximation, sampling, and finite-iteration optimization theory for variational physics-informed approximation of uniformly monotone nonlinear multiscale elliptic equations.
The paper introduces a two‑step debiased estimation method for PDE‑constrained inverse problems where the PDE solution is approximated by Physics‑Informed Neural Networks (PINNs). By combining neural‑network‑based nonparametric estimation with an influence‑function bias correction, the authors achieve a √{n}-consistent, asymptotically normal estimator without undersmoothing the neural network. The approach is extended to Bayesian inference, yielding a posterior that contracts at the √{n}-rate with asymptotic covariance matching the frequentist estimator, and the analysis also provides near‑minimax rates for estimating nonparametric regression functions and their derivatives in Sobolev spaces.
arXiv:2505. 07124v3 Announce Type: replace Abstract: We study inverse problems where an unknown potential is observed only through samples from the measure it induces by a convex variational principle.
arXiv:2606. 02078v1 Announce Type: new Abstract: The existing optimizers for deep neural networks (DNNs) typically rely on either the $\ell_2$ norm or the $\ell_\infty$ norm, resulting in optimizers that do not adapt well to substantial changes in curvature across parameter dimensions.