arXiv Machine Learning

Same Loss, Different Gradients

arXiv Machine Learning
2d ago

How Accurate Is Accurate Enough?

arXiv:2609.38785v1 Announce Type: new Abstract: How accurate must a numerical approximation be within a learning system? Primitive error alone cannot answer this question: errors of the same magnitud...

By Ningkang Peng, Qianfeng Yu, Jingyang Mao, Xiaoqian Peng, Yanhui Gu
arXiv Machine Learning
Sep 4

A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations

arXiv:2609. 03129v1 Announce Type: cross Abstract: Several classical machine-learning methods, such as KRRs and SVRs, are both computationally and analytically tractable since their estimators either admit closed-form expressions or are obtained by minimizing convex training objectives; neither feature is generally available for deep neural networks.

By Ruiyang Hong, Hrad Ghoukasian, Anastasis Kratsios
arXiv Machine Learning
Sep 15

Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels

The paper studies operator learning on function spaces using encoder–decoder architectures. It shows that as input and output resolutions grow, the induced kernels converge to a limiting kernel, enabling regularity assumptions independent of resolution. The authors derive upper and lower bounds for regularized stochastic gradient descent, extend the analysis to neural networks via the limiting neural tangent kernel, and provide error bounds and complexity guarantees for various kernel and encoding constructions.

By Lei Shi, Jia-Qi Yang, Ding-Xuan Zhou
arXiv Machine Learning
Jun 30

Learning from samples: inverse problems over measures

arXiv:2505. 07124v3 Announce Type: replace Abstract: We study inverse problems where an unknown potential is observed only through samples from the measure it induces by a convex variational principle.

By Francisco Andrade, Gabriel Peyr\'e, Clarice Poon
arXiv Machine Learning
Jun 25

A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients

arXiv:2606. 24999v1 Announce Type: new Abstract: High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging.

By Yanwei Jia, Du Ouyang, Huy\^en Pham, Xun Yu Zhou
arXiv Machine Learning
Jun 16

Functional Gradient Descent with Adaptive Representations

arXiv:2606. 16926v1 Announce Type: cross Abstract: Functional optimization problems are typically solved by optimizing the parameters of a fixed representation, such as a neural network, resulting in highly nonconvex losses that complicate both training and theoretical analysis.

By Daniel Csillag, Rodrigo Schuller, Pedro Dall'Antonia, Leonidas Guibas, Luiz Velho, Tiago Novello
arXiv Machine Learning
Jul 23

Optimal Recalibration of an Online Predictor

arXiv:2607. 19689v1 Announce Type: cross Abstract: We study the problem of recalibrating an online predictor [KE17, OKS24]: given an arbitrary "hint" sequence of forecasts, the learner must output new predictions that are calibrated while incurring small excess error relative to the original forecasts, under a proper loss.

By Lunjia Hu, Kevin Tian, Chutong Yang
arXiv AI
Aug 6

The Hamilton-Jacobi Theory of Deep Learning

arXiv:2605. 28983v2 Announce Type: replace-cross Abstract: In this paper, training a neural network is identified, exactly, as a search through Hamilton--Jacobi initial-value problems: each gradient step selects the initial data of a viscous Hamilton--Jacobi equation whose Hopf--Cole propagator best fits the observations; at inference, the input is the spatial point at which that solution is evaluated and the initial condition is already encoded in the weights.

By Jose Marie Antonio Mi\~noza, Erika Fille T. Legara, Christopher P. Monterola