The paper develops a mathematical theory of superposition in neural networks using frame theory and compressed sensing. It shows that a sparse binary vector of active features can be encoded by an overcomplete dictionary and recovered via a ReLU operation with a suitable bias. The authors prove recovery theorems for both random-support and worst-case support settings, providing high-probability guarantees for low-coherence dictionaries and a sharp criterion for sparsity levels, with explicit results for Gaussian random matrices and equiangular tight frames.
By Michael I. Ivanitskiy, John Jasper, Emily J. King, Dustin G. Mixon
arXiv:2510. 07758v3 Announce Type: replace Abstract: Sharpness (of the loss minima) is widely believed to be a good indicator of generalization of neural networks.
By Qiaozhe Zhang, Jun Sun, Ruijie Zhang, Yingzhuang Liu
The article surveys how the problem of extracting a common component from multiple jointly observed signals has been approached independently by statisticians, information theorists, and engineers. It reviews canonical correlation analysis, common information concepts, and subspace methods, then unifies these perspectives, extending to nonlinear CCA and discussing identifiability and links to self‑supervised learning. The tutorial aims to provide a self‑contained exposition that bridges signal processing, machine learning, and information theory.
By Paris A. Karakasis, Walid Saad
arXiv:2510. 24215v5 Announce Type: replace-cross Abstract: Recovery from linear measurements under sparse adversarial corruption is typically formulated as an exact-recovery problem: one seeks structural conditions on $\mathbf{A}$ (e.
By Vishal Halder, Alexandre Reiffers-Masson, Abdeldjalil A\"issa-El-Bey, Gugan Thoppe
arXiv:2512. 17473v3 Announce Type: replace-cross Abstract: We present an algorithm based on the alternating direction method of multipliers (ADMM) for solving nonlinear matrix decompositions (NMD).
By Atharva Awari, Nicolas Gillis, Arnaud Vandaele
arXiv:2607. 11938v1 Announce Type: cross Abstract: This book is about the mathematical foundations of data science.
By Afonso S. Bandeira, Amit Singer, Thomas Strohmer
arXiv:2609.06182v1 Announce Type: cross
Abstract: In this paper, we develop and analyze techniques for recovering a linear image $Bx$ of an unknown signal $x$ from indirect noisy observation $\omega=...
By Anatoli Juditsky, Arkadi Nemirovski
arXiv:2603.29715v2 Announce Type: replace
Abstract: Nonnegative matrix factorization (NMF) approximates a nonnegative matrix, X, by the product of two nonnegative factors, WH, where W has r columns a...
By Giovanni Seraghiti, K\'evin Dubrulle, Arnaud Vandaele, Nicolas Gillis
arXiv:2410. 01669v3 Announce Type: replace Abstract: Covariance Neural Networks (VNNs) perform graph convolutions on the covariance matrix of input data to leverage correlation information as pairwise connections.
By Andrea Cavallo, Zhan Gao, Elvin Isufi
arXiv:2609.32176v2 Announce Type: replace
Abstract: The support recovery problem in mixture of linear classifiers aims to identify the features relevant to the underlying decision rules when data is...
By Xiaxin Li, Arya Mazumdar
arXiv:2602. 04795v3 Announce Type: replace Abstract: Nonnegative matrix factorization (NMF) is a popular data embedding technique.
By Olivier Vu Thanh, Nicolas Gillis
arXiv:2512. 17426v2 Announce Type: replace-cross Abstract: We consider sparse signal reconstruction via minimization of the smoothly clipped absolute deviation (SCAD) penalty, and develop one-step replica-symmetry-breaking (1RSB) extensions of approximate message passing (AMP), termed 1RSB-AMP.
By Xiaosi Gu, Ayaka Sakata, Tomoyuki Obuchi