arXiv Machine Learning
Revenge of Monosemanticity: Specialized Neurons Improve Data Efficiency in MLPs
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 11
Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs
The paper investigates how multilayer perceptrons (MLPs) learn features in regression tasks with clustered data. It finds that instead of forming a single global low‑dimensional representation, MLPs develop monosemantic specialized neurons—each neuron aligns strongly with a specific predictive feature relevant to a particular region of the input space. This specialization results in a collection of local low‑dimensional representations, giving MLPs a provable data‑efficiency advantage over methods that rely on a global representation.
arXiv Machine Learning
Jul 28
A Statistical Difference between Single-Layer Learning and Hierarchical Learning in Wide Neural Networks
arXiv:2607. 23397v1 Announce Type: new Abstract: Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood.
arXiv Machine Learning
Jun 4
HalfNet: Randomized Neural Networks with Learned Subspace Geometry
arXiv:2606. 04583v1 Announce Type: new Abstract: Many researchers investigated neural networks with some of their weights fixed to values randomly drawn from a given distribution, e.
arXiv Machine Learning
Oct 19
Mesoscopic learning dynamics of extensive-width neural networks
arXiv:2610.23779v1 Announce Type: cross Abstract: Understanding how stochastic gradient descent learns internal representations in wide neural networks trained on rich target functions is a central p...
arXiv Machine Learning
Oct 2
Parameter symmetries determine representational geometry in overparameterized nonlinear networks
The paper investigates how parameter symmetries in overparameterized nonlinear neural networks affect representational geometry. It shows that these symmetries can be described by three basic feature transformations—addition, duplication, and scaling—allowing a closed‑form decomposition of representations into essential and auxiliary parts. The study further demonstrates that implementation‑level selection rules can resolve the resulting degeneracy, producing identifiable geometries that reflect feature contributions to the network’s function.
arXiv AI
Jun 8
A Geometric View for Understanding Concept Learning and Neuron Interpretation in Sparse Autoencoders
arXiv:2606. 07007v1 Announce Type: cross Abstract: We propose a unified mathematical framework for a geometric understanding of concept learning and neuron interpretation in sparse autoencoders (SAEs).