arXiv:2606. 00442v1 Announce Type: new Abstract: Many machine learning techniques rely on approximating a loss function's curvature, but this is notoriously hard to do at the scale of modern deep networks.
By Artem Artemev, Rui Xia, Benjamin M. Boyd, Youjing Yu, Felix Dangel, Guillaume Hennequin, Alberto Bernacchia
arXiv:2609.39020v1 Announce Type: new
Abstract: We study how well deep neural networks approximate and learn spectral Barron functions. Recent studies have shown that these function classes can be ef...
By Songqiu Ma, Yunfei Yang
The paper presents a theoretical analysis of symmetric autoencoders, a class of deep learning architectures frequently used in machine learning tasks. It distinguishes between different symmetric designs and shows that the reconstruction error of orthonormal symmetric autoencoders can be interpreted via the Eckart‑Young‑Schmidt theorem. Building on this insight, the authors propose an EYS‑based initialization strategy using repeated SVD, and validate its effectiveness through numerical experiments comparing it to conventional deep autoencoders.
By Simone Brivio, Nicola Rares Franco
arXiv:2503. 04263v2 Announce Type: replace Abstract: Motivated by applications to the simulation of quantum many-body systems by neural networks, researchers have suggested several models which are antisymmetric by construction, and can approximate all antisymmetric functions.
By Nadav Dym, Jianfeng Lu, Matan Mizrachi
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
The paper introduces Shallow Universal Polynomial Networks (SUPNs), a single‑hidden‑layer architecture that replaces standard activations with learnable multivariate polynomials, thereby increasing expressivity per unit. SUPNs achieve universal approximation by either widening the network or increasing polynomial complexity, and the authors prove convergence rates and quasi‑optimal parameter choices. Empirical tests across a wide range of smooth, nonsmooth, and discontinuous functions in up to ten dimensions show that SUPNs often outperform deep neural networks and Kolmogorov‑Arnold networks in accuracy and variability while requiring fewer parameters and lower training cost.
By Zachary Morrow, Michael Penwarden, Brian Chen, Aurya Javeed, Akil Narayan, John D. Jakeman