The paper investigates how spectral summaries and scalar learning curves can reveal properties of centered quadratic networks. By constructing analytic positive input densities with Gaussian one- and two-coordinate marginals, the authors show that certain teachers with a nonzero cycle invariant produce distinct factorized loss curves, while others yield identical anchored spectral measures. The study further explores the sample complexity required for noisy scalar loss queries and bounded initialization interventions, identifies conditions under which a single scalar loss curve can recover the initial predictor and teacher spectrum, and provides exact recovery guarantees for fixed dimensions and distinct teacher sums.
We study what spectral summaries and scalar learning curves reveal about centered quadratic networks. We construct analytic positive input densities with exactly Gaussian one- and two-coordinate margi...
arXiv:2609.39408v1 Announce Type: cross Abstract: Population loss can remain nearly constant while a neural network learns a substantially more predictive representation. We establish this separation...
arXiv:2608. 10172v1 Announce Type: new Abstract: Mechanistic interpretability explains models by identifying circuits inside them, but has no way to tell whether a circuit is a property of the model or an artifact of the method that found it.
arXiv:2608.29472v1 Announce Type: cross Abstract: Knowledge distillation trains a small student model to reproduce the outputs of a large teacher model, and its progress is typically monitored throug...
arXiv:2608. 15472v1 Announce Type: cross Abstract: The problem of networked information aggregation, studied in Kearns et al.
arXiv:2607. 25624v1 Announce Type: new Abstract: Positive quadratic networks admit the low-rank representation f_U(x)=x^top UU^top x, where Uinmathbb{R}^{dtimes r} is identifiable only up to right orthogonal multiplication, representing a rank-r PSD matrix Q=UU^top.
arXiv:2610. 09356v1 Announce Type: new Abstract: We prove global exponential (linear) convergence with an explicit rate in the rich scaling for wide two-layer linear networks trained with smooth Polyak-Lojasiewicz predictor losses.
arXiv:2610.27010v1 Announce Type: cross Abstract: We study robust estimation of a Gaussian precision matrix that decomposes as a sparse matrix minus a symmetric, possibly indefinite, low-rank matrix,...
arXiv:2609. 12994v1 Announce Type: new Abstract: Heavy-tailed empirical spectral densities of neural-network weight matrices are widely used as diagnostics of implicit self-regularization, but the step complexity of heavy-tail emergence remains poorly understood.
arXiv:2607. 19771v1 Announce Type: cross Abstract: Muon and related matrix-sign optimizers are increasingly used to pre-train large language models, but their effect on the internal geometry of individual weight matrices is not well understood.
arXiv:2510.22778v4 Announce Type: replace-cross Abstract: A growing class of machine-learning objects -- covariance and Gram matrices, kernel and attention matrices, MIMO channel matrices, density op...
arXiv:2610. 25116v2 Announce Type: replace Abstract: We study positive, bias-free, one-hidden-layer ReLU networks under standard Gaussian inputs.