← Back to all news
arXiv Machine Learning By Zach Furman, Stephan W\"aldchen, Yangda Bei, Liam Hodgkinson

Benign Loss Landscapes Can Coexist with Worst-Case Hardness

Read the original on arXiv Machine Learning →

The paper demonstrates that tree tensor networks (TTNs) can encode arbitrary read‑once Boolean formulas, yielding polynomial‑size targets that are hard for gradient descent to learn in polynomial time, yet their loss landscapes are conditionally benign: every minimum‑norm local minimum is global. This shows that bad local minima are not the source of learning difficulty in TTNs; instead, high‑order degenerate saddle points caused by rank‑deficiency can impede learning. A case study on the parity function illustrates how TTNs can link landscape geometry to computational hardness.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Aug 25

Which Algorithms Can Graph Neural Networks Learn?

arXiv:2602.13106v2 Announce Type: replace-cross Abstract: In recent years, there has been growing interest in understanding neural architectures' ability to learn to execute discrete algorithms, a li...

By Solveig Wittig, Antonis Vasileiou, Robert R. Nerem, Timo Stoll, Floris Geerts, Yusu Wang, Christopher Morris
efficiency
More like this →
arXiv Machine Learning
Oct 16

SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximation

The paper introduces Shallow Universal Polynomial Networks (SUPNs), a single‑hidden‑layer architecture that replaces standard activations with learnable multivariate polynomials, thereby increasing expressivity per unit. SUPNs achieve universal approximation by either widening the network or increasing polynomial complexity, and the authors prove convergence rates and quasi‑optimal parameter choices. Empirical tests across a wide range of smooth, nonsmooth, and discontinuous functions in up to ten dimensions show that SUPNs often outperform deep neural networks and Kolmogorov‑Arnold networks in accuracy and variability while requiring fewer parameters and lower training cost.

By Zachary Morrow, Michael Penwarden, Brian Chen, Aurya Javeed, Akil Narayan, John D. Jakeman
More like this →
arXiv Machine Learning
Oct 19

Neural Operators Which Certifiably Simultaneously Solve Infinitely Many Convex Programs using Logarithmic Rank and Linear Complexity

The paper introduces proximal equilibrium operators (PEOs), a class of neural operators that can simultaneously solve infinitely many convex optimization problems over a separable Hilbert space. A short training algorithm is presented that constructs PEOs using finite‑dimensional deep equilibrium layers, and a theoretical guarantee shows that, for inputs from a suitable infinite‑dimensional set, the trained PEOs uniformly approximate the solutions to arbitrary precision with logarithmic rank and width and linear depth in the inverse error. The authors validate their approach on applications to nonlinear PDEs, stochastic optimal control, and portfolio optimization and hedging in mathematical finance.

By Anastasis Kratsios, Ariel Neufeld, Philipp Schmocker
More like this →