arXiv Machine Learning By Anastasis Kratsios, Ariel Neufeld, Philipp Schmocker

Neural Operators Which Certifiably Simultaneously Solve Infinitely Many Convex Programs using Logarithmic Rank and Linear Complexity

Read the original on arXiv Machine Learning →

The paper introduces proximal equilibrium operators (PEOs), a class of neural operators that can simultaneously solve infinitely many convex optimization problems over a separable Hilbert space. A short training algorithm is presented that constructs PEOs using finite‑dimensional deep equilibrium layers, and a theoretical guarantee shows that, for inputs from a suitable infinite‑dimensional set, the trained PEOs uniformly approximate the solutions to arbitrary precision with logarithmic rank and width and linear depth in the inverse error. The authors validate their approach on applications to nonlinear PDEs, stochastic optimal control, and portfolio optimization and hedging in mathematical finance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Sep 24

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon

The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.

By Hao Yu
arXiv Machine Learning
Sep 18

The First-Order Oracle Complexity of Lipschitz Convex Optimization in Nondual Settings

arXiv:2609. 20687v1 Announce Type: cross Abstract: We study first-order black-box convex optimization over an $\ell_p$-ball for objectives Lipschitz in the $\ell_q$-norm, solving in the affirmative the nonsmooth version of the COLT open question (Guz15b) on whether the geometry of a smaller feasible set ($p < q$) can improve convergence rates in convex optimization, and matching prior lower bounds up to logarithmic factors.

By David Mart\'inez-Rubio, Brian Bullins, Crist\'obal Guzm\'an, Mathieu Molina