arXiv Machine Learning

On the robustness of noisy solutions in non-convex neural networks

arXiv:2607. 27000v1 Announce Type: cross Abstract: Optimization in non-convex neural network models is strongly influenced by the geometry of the solution space: sparse, isolated, point-like clusters are typically algorithmically inaccessible, whereas wide and flat regions can be found efficiently despite being relatively rare.

Hugging Face Trending Papers

Aug 3

Convex Neural Energy Elements: Monolithic Finite-Element Assembly of Geometry-Parameterized Neural Operators with Stability and Error Guarantees

Extending the neural-operator element method from individually trained, fixed-geometry neural elements to a library of reusable, geometry-parameterized element types fails structurally: a field-predicting operator trained by value regression induces an energy whose assembled Hessian is indefinite, and Newton converges to spurious minima (247% error) even with 1%-accurate field predictions. We introduce convex neural energy elements: each element exports a scalar energy E(g,U), architecturally convex in its boundary degrees of freedom U and smoothly parameterized by its geometry g, realized as a hypernetwork-generated positive-semidefinite quadratic form (an input-convex correction is reserved for non-quadratic physics).

safety

arXiv Machine Learning

Jun 30

Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model

arXiv:2606. 28573v1 Announce Type: new Abstract: Modern machine learning models are trained by optimizing high-dimensional non-convex empirical risk functions.

By Andrea Montanari, Kangjie Zhou

arXiv Machine Learning

Jul 27

Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolation

arXiv:2501. 18530v3 Announce Type: replace-cross Abstract: We consider a teacher-student model of supervised learning with a fully-trained two-layer neural network whose width $k$ and input dimension $d$ are large and proportional.

By Jean Barbier, Francesco Camilli, Minh-Toan Nguyen, Mauro Pastore, Rudy Skerk

safety

arXiv Machine Learning

Aug 4

Convex Neural Energy Elements: Monolithic Finite-Element Assembly of Geometry-Parameterized Neural Operators with Stability and Error Guarantees

arXiv:2608. 02036v1 Announce Type: new Abstract: Extending the neural-operator element method from individually trained, fixed-geometry neural elements to a library of reusable, geometry-parameterized element types fails structurally: a field-predicting operator trained by value regression induces an energy whose assembled Hessian is indefinite, and Newton converges to spurious minima (247% error) even with 1%-accurate field predictions.

By Hongyue Jiang, Jianjiang Zhan, Chenzhuo Zhang, Fan Wang

safety

arXiv Machine Learning

Jul 20

A Statistical Formulation Gap for Nonlinear Multiscale Physics-Informed Learning

arXiv:2607. 15702v1 Announce Type: cross Abstract: We prove a finite-sample formulation gap for physics-informed learning of nonlinear multiscale elliptic equations.

By Ronald Katende

diffusion

arXiv Machine Learning

Aug 6

Variational Bounds for Perceptron Learning from Structured Data

arXiv:2608. 04882v1 Announce Type: new Abstract: We introduce a variational approach to a finite-temperature continuous-spin perceptron trained on a Gaussian mixture.

By Francesco Camilli, Pierluigi Contucci, Federica Gerace, Emanuele Mingione

arXiv Machine Learning

Jul 23

Non-Asymptotic Variational Learning for Monotone Nonlinear Multiscale Elliptic Equations: Scale-Robust Primal-Dual Bounds and Strong-Form Statistical Ill-Conditioning

arXiv:2607. 15702v2 Announce Type: replace-cross Abstract: We develop a non-asymptotic approximation, sampling, and finite-iteration optimization theory for variational physics-informed approximation of uniformly monotone nonlinear multiscale elliptic equations.

By Ronald Katende

arXiv Machine Learning

1d ago

DeepOHeat-v2: Self-Improving Operator Learning for Fast and Trustworthy Thermal Optimization in 3D-IC Design

arXiv:2608. 16080v1 Announce Type: new Abstract: Thermal-aware optimization of multi-die 3D integrated circuits evaluates many designs, each a costly heat-equation solve.

By Xinling Yu, Yixing Li, Ziyue Liu, Xin Ai, Zhiyu Zeng, Hai Li, Zheng Zhang

benchmarks

arXiv Machine Learning

Jul 27

Statistical mechanics of extensive-width Bayesian neural networks near interpolation

arXiv:2505. 24849v2 Announce Type: replace-cross Abstract: For three decades statistical mechanics has been providing a framework to analyse neural networks.

By Jean Barbier, Francesco Camilli, Minh-Toan Nguyen, Mauro Pastore, Rudy Skerk

arXiv Machine Learning

5d ago

Training Under Challenge: Executable Certificates and Challenge-Closed Optimality for Neural Networks

arXiv:2608. 12655v1 Announce Type: new Abstract: A flat training curve does not reveal whether a neural network has reached a global optimum, is locally trapped, is representation-limited, or is mismatched to its trainer.

By Farhang Yeganegi, Arian Eamaz, Mojtaba Soltanalian

efficiency

arXiv Machine Learning

Jun 19

How to sketch a learning algorithm

arXiv:2604. 07328v3 Announce Type: replace Abstract: How does the choice of training data influence an AI model?

By Sam Gunn

safety

arXiv Machine Learning

Jun 30

Why Do We Need Warm-up? A Theoretical Perspective

arXiv:2510. 03164v2 Announce Type: replace Abstract: Learning rate warm-up -- increasing the learning rate at the beginning of training -- has become a ubiquitous heuristic in modern deep learning, yet its theoretical foundations remain poorly understood.

By Foivos Alimisis, Rustem Islamov, Aurelien Lucchi

Related stories