arXiv Machine Learning
Jun 19
Recurrent neural networks approximate continuous functions
arXiv:2606. 20325v1 Announce Type: new Abstract: Classical approximation theorems ask for a new neural network whenever the target accuracy is improved.
Compiling to recurrent neurons
arXiv:2511. 14953v2 Announce Type: replace-cross Abstract: Discrete structures are currently second-class in differentiable programming.
Related stories
arXiv Machine Learning
Jun 17
A Dynamical Systems Perspective on the Analysis of Neural Networks
arXiv:2507. 05164v2 Announce Type: replace-cross Abstract: In this chapter, we utilize dynamical systems to analyze several aspects of machine learning algorithms.
arXiv AI
Jun 9
MinMax Recurrent Neural Cascades
arXiv:2605. 06384v3 Announce Type: replace-cross Abstract: We introduce MinMax Recurrent Neural Cascades (MinMax RNCs), a class of recurrent neural networks built from a novel form of recurrence over the MinMax algebra.
arXiv AI
Aug 25
Which Algorithms Can Graph Neural Networks Learn?
arXiv:2602.13106v2 Announce Type: replace-cross Abstract: In recent years, there has been growing interest in understanding neural architectures' ability to learn to execute discrete algorithms, a li...
arXiv AI
Aug 17
Breaking Chains with Trees: Model-Parallel Deep Learning with $\mathcal{O}(\log N)$ Time Complexity
arXiv:2606. 21497v2 Announce Type: replace-cross Abstract: Modern deep neural networks are trained using error backpropagation, which requires sequential forward and backward computations across network layers.
arXiv Machine Learning
Aug 4
Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions
arXiv:2602. 06737v2 Announce Type: replace Abstract: We present a generalized framework for the range verification of neural networks featuring non-linear activation functions.
arXiv Machine Learning
Aug 11
Input convex neural networks as surrogates in mathematical optimisation
arXiv:2608. 09707v1 Announce Type: cross Abstract: Embedding trained neural networks as surrogates within optimisation problems is an established practice in operations research.
arXiv AI
Jun 26
Algorithmic Foundations of Deep Learning: Complexity-Theoretic Rates and a Characterization of Universal Approximation
arXiv:2606. 26705v1 Announce Type: cross Abstract: Feedforward neural network (NN) expressivity is typically studied by emulating optimal basis-expansion schemes.
arXiv Machine Learning
Sep 14
Benign Loss Landscapes Can Coexist with Worst-Case Hardness
The paper demonstrates that tree tensor networks (TTNs) can encode arbitrary read‑once Boolean formulas, yielding polynomial‑size targets that are hard for gradient descent to learn in polynomial time, yet their loss landscapes are conditionally benign: every minimum‑norm local minimum is global. This shows that bad local minima are not the source of learning difficulty in TTNs; instead, high‑order degenerate saddle points caused by rank‑deficiency can impede learning. A case study on the parity function illustrates how TTNs can link landscape geometry to computational hardness.
arXiv Machine Learning
Jun 30
Negative Stepsizes Make Gradient-Descent-Ascent Converge
arXiv:2505. 01423v2 Announce Type: replace-cross Abstract: Efficient computation of min-max problems is a central question in optimization, learning, games, and control.
arXiv AI
Jul 29
CIFNet: An Analytic Neural Learning Framework for Efficient and Calibrated Class-Incremental Learning
arXiv:2509. 11285v2 Announce Type: replace-cross Abstract: Class-Incremental Learning (CIL) in deep neural networks is conventionally framed as an iterative gradient-based optimization problem, incurring high computational cost, hyperparameter sensitivity, and risk of catastrophic forgetting.
arXiv AI
Jul 21
Emergent Hierarchical Monosemantic Neurons from the Group-Contrastive Forward-Forward Algorithm
arXiv:2607. 16295v1 Announce Type: cross Abstract: Mechanistic interpretability has made significant strides in understanding neural network representations, with sparse dictionary learning (SDL) methods, most prominently sparse autoencoders, as a central paradigm.