arXiv Machine Learning
Jul 21
The Value of Depth in Message Passing on Sparse Graphs: A Kesten-Stigum Dichotomy
arXiv:2607. 16676v1 Announce Type: cross Abstract: How deep does a graph neural network need to be on a sparse graph?
From Approximation Rates to Loss-Landscape Barrier Decay in Shallow ReLU Networks
arXiv:2602. 17596v2 Announce Type: replace Abstract: We study pathwise connectivity of sublevel sets for one-hidden-layer ReLU networks with constrained first-layer weights and an $\ell_1$ penalty on the output layer.
Related stories
arXiv Machine Learning
5d ago
Gradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation
arXiv:2606. 21253v2 Announce Type: replace Abstract: Continual learning that is gradient-free, local, online, and append-only is attractive for edge and streaming deployment, but its value is usually argued informally.
arXiv Machine Learning
5d ago
Training Non-Differentiable Networks via Optimal Transport
arXiv:2605. 01928v2 Announce Type: replace Abstract: We optimize losses that jump: spiking thresholds, quantized layers, and discrete routing put jumps in the forward pass, where backpropagation does not apply.
arXiv Machine Learning
Jul 10
A law of robustness for two-layer neural networks with arbitrary weights
arXiv:2607. 07778v1 Announce Type: new Abstract: Bubeck, Li and Nagaraj conjectured that, for generic data, any two-layer neural network with $m$ neurons that fits $n$ noisy labels must have Lipschitz constant at least of order $\sqrt{n/m}$, with no restriction on the size of the weights.
arXiv Machine Learning
Jul 8
Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations
arXiv:2607. 05815v1 Announce Type: new Abstract: The Minkowski functionals of a field's excursion sets -- area, boundary measure, and Euler characteristic -- describe its level-set morphology; the Euler characteristic is the cheapest handle on topology.
arXiv Machine Learning
Jun 24
Layer-wise Geometric Approximation Rates for Deep Networks
arXiv:2604. 20219v2 Announce Type: replace Abstract: Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for the final output and leaves the role of intermediate layers largely unclear.
arXiv Machine Learning
Jul 9
Avoiding unsafe sets when training with Langevin Dynamics
arXiv:2607. 07538v1 Announce Type: new Abstract: Training a model with noisy gradient descent can be idealized as overdamped Langevin dynamics on the loss landscape, and a natural safety question is to bound the probability $\nu_t(\mathcal{A}_H) = \mathbb{P}(Q_t \in \mathcal{A}_H)$ that the trajectory lies in a designated failure region $\mathcal{A}_H$.
arXiv Machine Learning
Jul 17
Random Parameter Noise Does Not Make Exact ReLU Verification Easy
arXiv:2607. 14375v1 Announce Type: cross Abstract: We study exact verification of ReLU networks in an adversarial smoothed model.
arXiv Machine Learning
5d ago
Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich
arXiv:2608. 13201v1 Announce Type: cross Abstract: We develop the statistical and algorithmic theory of inverse optimal transport (IOT) under the feature-parameterized cost C_theta(i,j) = -theta^T phi(i,j).
arXiv Machine Learning
Jul 23
Non-Asymptotic Variational Learning for Monotone Nonlinear Multiscale Elliptic Equations: Scale-Robust Primal-Dual Bounds and Strong-Form Statistical Ill-Conditioning
arXiv:2607. 15702v2 Announce Type: replace-cross Abstract: We develop a non-asymptotic approximation, sampling, and finite-iteration optimization theory for variational physics-informed approximation of uniformly monotone nonlinear multiscale elliptic equations.
arXiv Machine Learning
Jun 15
Nonlinear Two-Time-Scale Stochastic Approximation: A Sharp Phase Transition and How to Beat It
arXiv:2606. 14488v1 Announce Type: cross Abstract: Recent finite-time analyses of nonlinear two-time-scale stochastic approximation show that under contractive assumptions the slow iterate $Y_k$ with stepsizes $\beta_k=\Theta(k^{-1})$ and $\alpha_k=\Theta(k^{-a})$, $a\in(1/2,1)$, generally satisfies a mean-square rate of order $k^{-a}$; decoupled $k^{-1}$ rates require strong local linearity.
arXiv Machine Learning
Jun 15
Recovery thresholds for hidden weighted sparse graphs
arXiv:2606. 14335v1 Announce Type: cross Abstract: Recovering structural information from noisy high-dimensional data is a fundamental task in statistical inference.