arXiv Machine Learning
Revenge of Monosemanticity: Specialized Neurons Improve Data Efficiency in MLPs
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 11
Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs
The paper investigates how multilayer perceptrons (MLPs) learn features in regression tasks with clustered data. It finds that instead of forming a single global low‑dimensional representation, MLPs develop monosemantic specialized neurons—each neuron aligns strongly with a specific predictive feature relevant to a particular region of the input space. This specialization results in a collection of local low‑dimensional representations, giving MLPs a provable data‑efficiency advantage over methods that rely on a global representation.
arXiv Machine Learning
Jul 28
A Statistical Difference between Single-Layer Learning and Hierarchical Learning in Wide Neural Networks
arXiv:2607. 23397v1 Announce Type: new Abstract: Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood.
arXiv Machine Learning
Jun 4
HalfNet: Randomized Neural Networks with Learned Subspace Geometry
arXiv:2606. 04583v1 Announce Type: new Abstract: Many researchers investigated neural networks with some of their weights fixed to values randomly drawn from a given distribution, e.
arXiv AI
Jun 8
A Geometric View for Understanding Concept Learning and Neuron Interpretation in Sparse Autoencoders
arXiv:2606. 07007v1 Announce Type: cross Abstract: We propose a unified mathematical framework for a geometric understanding of concept learning and neuron interpretation in sparse autoencoders (SAEs).
arXiv AI
Aug 12
Emergent Neural Network Mechanisms for Generalization to Objects in Novel Orientations
arXiv:2109. 13445v3 Announce Type: replace-cross Abstract: The capability of Deep Neural Networks (DNNs) to recognize objects in orientations outside the distribution of the training data is not well understood.
arXiv Machine Learning
Sep 10
Feature Superposition in Neural Networks: From Theory to Practice
arXiv:2609.06862v1 Announce Type: new Abstract: Superposition refers to neural networks representing more features than they have dimensions. It offers a possible explanation for polysemantic neurons...