← Back to all news
arXiv Machine Learning By Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis, Mikhail Belkin

Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs

Read the original on arXiv Machine Learning →

The paper investigates how multilayer perceptrons (MLPs) learn features in regression tasks with clustered data. It finds that instead of forming a single global low‑dimensional representation, MLPs develop monosemantic specialized neurons—each neuron aligns strongly with a specific predictive feature relevant to a particular region of the input space. This specialization results in a collection of local low‑dimensional representations, giving MLPs a provable data‑efficiency advantage over methods that rely on a global representation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 2

Parameter symmetries determine representational geometry in overparameterized nonlinear networks

The paper investigates how parameter symmetries in overparameterized nonlinear neural networks affect representational geometry. It shows that these symmetries can be described by three basic feature transformations—addition, duplication, and scaling—allowing a closed‑form decomposition of representations into essential and auxiliary parts. The study further demonstrates that implementation‑level selection rules can resolve the resulting degeneracy, producing identifiable geometries that reflect feature contributions to the network’s function.

By Marvin Theiss, Lukas Braun, Andrew M. Saxe, Erin Grant
More like this →