The paper develops a mathematical theory of superposition in neural networks using frame theory and compressed sensing. It shows that a sparse binary vector of active features can be encoded by an overcomplete dictionary and recovered via a ReLU operation with a suitable bias. The authors prove recovery theorems for both random-support and worst-case support settings, providing high-probability guarantees for low-coherence dictionaries and a sharp criterion for sparsity levels, with explicit results for Gaussian random matrices and equiangular tight frames.
By Michael I. Ivanitskiy, John Jasper, Emily J. King, Dustin G. Mixon
arXiv:2510. 07758v3 Announce Type: replace Abstract: Sharpness (of the loss minima) is widely believed to be a good indicator of generalization of neural networks.
By Qiaozhe Zhang, Jun Sun, Ruijie Zhang, Yingzhuang Liu
The article surveys how the problem of extracting a common component from multiple jointly observed signals has been approached independently by statisticians, information theorists, and engineers. It reviews canonical correlation analysis, common information concepts, and subspace methods, then unifies these perspectives, extending to nonlinear CCA and discussing identifiability and links to self‑supervised learning. The tutorial aims to provide a self‑contained exposition that bridges signal processing, machine learning, and information theory.
By Paris A. Karakasis, Walid Saad
arXiv:2510. 24215v5 Announce Type: replace-cross Abstract: Recovery from linear measurements under sparse adversarial corruption is typically formulated as an exact-recovery problem: one seeks structural conditions on $\mathbf{A}$ (e.
By Vishal Halder, Alexandre Reiffers-Masson, Abdeldjalil A\"issa-El-Bey, Gugan Thoppe
arXiv:2512. 17473v3 Announce Type: replace-cross Abstract: We present an algorithm based on the alternating direction method of multipliers (ADMM) for solving nonlinear matrix decompositions (NMD).
By Atharva Awari, Nicolas Gillis, Arnaud Vandaele
arXiv:2607. 11938v1 Announce Type: cross Abstract: This book is about the mathematical foundations of data science.
By Afonso S. Bandeira, Amit Singer, Thomas Strohmer