Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs

The paper investigates how multilayer perceptrons (MLPs) learn features in regression tasks with clustered data. It finds that instead of forming a single global low‑dimensional representation, MLPs develop monosemantic specialized neurons—each neuron aligns strongly with a specific predictive feature relevant to a particular region of the input space. This specialization results in a collection of local low‑dimensional representations, giving MLPs a provable data‑efficiency advantage over methods that rely on a global representation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.