The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper investigates how spectral summaries and scalar learning curves can reveal properties of centered quadratic networks. By constructing analytic positive input densities with Gaussian one- and two-coordinate marginals, the authors show that certain teachers with a nonzero cycle invariant produce distinct factorized loss curves, while others yield identical anchored spectral measures. The study further explores the sample complexity required for noisy scalar loss queries and bounded initialization interventions, identifies conditions under which a single scalar loss curve can recover the initial predictor and teacher spectrum, and provides exact recovery guarantees for fixed dimensions and distinct teacher sums.
arXiv:2609.39408v1 Announce Type: cross Abstract: Population loss can remain nearly constant while a neural network learns a substantially more predictive representation. We establish this separation...
arXiv:2608.29472v1 Announce Type: cross Abstract: Knowledge distillation trains a small student model to reproduce the outputs of a large teacher model, and its progress is typically monitored throug...
arXiv:2608. 10172v1 Announce Type: new Abstract: Mechanistic interpretability explains models by identifying circuits inside them, but has no way to tell whether a circuit is a property of the model or an artifact of the method that found it.
arXiv:2609.37745v1 Announce Type: new Abstract: Neural scaling, in which loss falls as a power law with training, is central to large language models, and one recent proposal is that a $1/3$ exponent...
arXiv:2609. 12994v1 Announce Type: new Abstract: Heavy-tailed empirical spectral densities of neural-network weight matrices are widely used as diagnostics of implicit self-regularization, but the step complexity of heavy-tail emergence remains poorly understood.