← Back to all news
arXiv Machine Learning By Sejun Park, Yeachan Park, Geonho Hwang

Floating-Point Networks with Automatic Differentiation Can Represent Almost All Floating-Point Functions and Their Gradients

Read the original on arXiv Machine Learning →

arXiv:2605. 01702v2 Announce Type: replace Abstract: Theoretical studies show that for any differentiable function on a compact domain, there exists a neural network that approximates both the function values and gradients.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jul 16

Automatic Differentiation from Scratch: How PyTorch Computes Gradients in Physics-Informed Neural Networks

arXiv:2607. 13042v1 Announce Type: new Abstract: This paper traces, with explicit numerical values, how PyTorch's automatic differentiation (AD) engine computes gradients for Physics-Informed Neural Network (PINN) training -- a setting that requires two levels of differentiation: computing the physics derivative $\hat{y}'(t)=d\hat{y}/dt$ through the network, and computing parameter gradients $\nabla_\theta L$ of a loss that itself depends on $\hat{y}'(t)$.

By Abdeladhim Tahimi
More like this →
arXiv Machine Learning
Jun 16

Functional Gradient Descent with Adaptive Representations

arXiv:2606. 16926v1 Announce Type: cross Abstract: Functional optimization problems are typically solved by optimizing the parameters of a fixed representation, such as a neural network, resulting in highly nonconvex losses that complicate both training and theoretical analysis.

By Daniel Csillag, Rodrigo Schuller, Pedro Dall'Antonia, Leonidas Guibas, Luiz Velho, Tiago Novello
computer-vision
More like this →
Hugging Face Trending Papers
Aug 11

Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond

We systematically investigate finite-difference (FD) derivative computation in Physics-Informed Neural Networks (PINNs) as an alternative to automatic differentiation (AD). On three benchmark PDEs we show that, with a properly calibrated step size, FD matches AD in accuracy on every problem while running faster across the full tested batch-size range and using substantially less GPU memory, and that a stochastic variant we propose outperforms AD on a stationary problem.

benchmarks
More like this →
arXiv Machine Learning
Jul 14

Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width

arXiv:2607. 10589v1 Announce Type: cross Abstract: In contrast to most studies on neural network approximation theory that characterize results through a single parameter, such as the total number of network parameters, \cite{shen2020deep} pioneered the characterization of approximation rates as a joint function of the width parameter $N$ and the depth parameter $L$, thereby granting greater architectural flexibility.

By Yanming Lai, Defeng Sun, Yang Wang
More like this →