arXiv Machine Learning

The Price of Sparsity: Sufficient Conditions for Sparse Recovery using Sparse and Sparsified Measurements

arXiv Machine Learning
Jul 14

Demixing Sparse Signals from Nonlinear Observations using Generalized Non-convex Regularization

arXiv:2607. 10618v1 Announce Type: cross Abstract: We consider the recovery of a pair of sparse vectors from a limited number of nonlinear observations of their superposition: $y_i=g(\inner{\ba_i}{\bPhi\bw^\ast+\bPsi\bz^\ast})+e_i$, $i=1,\dots,m$, with $m\ll n$, incoherent orthonormal bases $\bPhi,\bPsi$, a scalar link $g$, and noise $e_i$ that may be heavy-tailed or contaminated.

By Raziyeh Takbiri
arXiv Machine Learning
Sep 4

Restricted Eigenvalues Beyond Gaussian Width: Threshold Occupancy under Heavy Tails

The paper investigates restricted eigenvalue (RE) bounds for norm‑regularized estimators under heavy‑tailed designs. It shows that the previously conjectured sample‑size law based on Gaussian width fails for heavy‑tailed measurements, due to a phenomenon called simultaneous threshold occupancy. The authors provide explicit counterexamples, derive worst‑case sample‑complexity bounds, and compare the behavior of heavy‑tailed versus Gaussian designs on constant‑width polyhedral descent cones.

By Shi Fu, Huibo Xu, Qixin Zhang, Dacheng Tao
arXiv Machine Learning
Aug 31

Towards a mathematical theory of superposition

The paper develops a mathematical theory of superposition in neural networks using frame theory and compressed sensing. It shows that a sparse binary vector of active features can be encoded by an overcomplete dictionary and recovered via a ReLU operation with a suitable bias. The authors prove recovery theorems for both random-support and worst-case support settings, providing high-probability guarantees for low-coherence dictionaries and a sharp criterion for sparsity levels, with explicit results for Gaussian random matrices and equiangular tight frames.

By Michael I. Ivanitskiy, John Jasper, Emily J. King, Dustin G. Mixon