arXiv Machine Learning

Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices

arXiv:2608. 03368v1 Announce Type: new Abstract: For $n$ unit vectors $x_1,\ldots,x_n \in \mathbb{R}^d$, we study the continuous ReLU derivative Gram matrix $H$, whose entries are obtained by averaging pairwise gated inner products over a standard Gaussian direction.

arXiv Machine Learning
Jul 13

A Fourier analytique approach to Gaussian mixture learning

arXiv:2004. 05813v3 Announce Type: replace-cross Abstract: Suppose that we are given independent, identically distributed random samples $x_1,\cdots,x_n$ from a mixture at most $k$ many $d$-dimensional spherical Gaussian distributions $\mu_1,\cdots,\mu_{k_0}$ of identical and known variance $\sigma^2$ in each coordinate, such that the minimum $\ell^2$ distance between two distinct centers $y_l$ and $y_j$ is greater than $2\Delta\sigma \min\{\sqrt{d},\sqrt k\}$, where $\Delta>C_0$, and $C_0$ is a sufficiently large universal constant.

By Somnath Chakraborty, Hariharan Narayanan
arXiv Machine Learning
Jul 28

Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD

arXiv:2607. 24235v1 Announce Type: cross Abstract: Over the past 20 years, kernel discrepancies have been leveraged as a highly powerful tool for quantifying the disagreement of distributions, with numerous successful applications in two-sample, goodness-of-fit, and independence testing, among others.

By Jose Cribeiro-Ramallo, Florian Kalinke, Zolt\'an Szab\'o
arXiv Machine Learning
Jul 28

A Resolution of the SS--RS--GD Inequalities

arXiv:2607. 22620v1 Announce Type: cross Abstract: Yun, Sra, and Jadbabaie (COLT 2021, open question) conjectured the SS--RS--GD inequalities: for well-conditioned symmetric matrices $A_1,\dots,A_n$, the operators $W_{ss}$, $W_{rs}$, and $W_{gd}$ that encode the expected iterate of single-shuffle SGD, random-reshuffle SGD, and gradient descent on a quadratic finite sum should satisfy \[ \|W_{ss}\|\le \| W_{rs}\|\le \|W_{gd}\|.

By Binghui Peng
arXiv Machine Learning
Jun 16

Phase Transition in Convex Relaxations for Graph Alignment

arXiv:2606. 15581v1 Announce Type: cross Abstract: We study the graph alignment problem for correlated Gaussian Orthogonal Ensemble (GOE) matrices, where the goal is to recover a hidden vertex permutation given two correlated symmetric Gaussian matrices $(A, B)$ with correlation $1/\sqrt{1+\sigma^2}$.

By Laurent Massouli\'e, Sushil Mahavir Varma, Louis Vassaux, Ir\`ene Waldspurger
arXiv Machine Learning
Aug 10

Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

arXiv:2608. 06687v1 Announce Type: cross Abstract: We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations $\mathfrak L_\beta u=f$ using linearized ReLU$^k$ neural networks on the sphere, where $\mathfrak L_\beta$ is a positive elliptic spectral multiplier of order $\beta$.

By Xinliang Liu, Tong Mao, Jinchao Xu