arXiv Machine Learning June 18, 2026 By Mriganka Basu Roy Chowdhury, Eric McLaughlin Weiner

Effects of sparsity and superposition on loss in simple autoencoders

Read the original on arXiv Machine Learning →

arXiv:2606. 18538v1 Announce Type: new Abstract: One of the major difficulties in the mechanistic interpretability of neural networks is the occurrence of polysemanticity, which suggests that each neuron is typically responsible for multiple different tasks, impeding a clean interpretation of their function.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 7

Compressed Computation under $L^4$ Loss is likely Computation in Superposition

arXiv:2607. 04800v1 Announce Type: new Abstract: Neural networks are thought to represent concepts as directions in their activation space, and superposition lets them encode more concepts than they have dimensions.

By Francisco Ferreira da Silva, Stefan Heimersheim

arXiv Machine Learning

Sep 10

Feature Superposition in Neural Networks: From Theory to Practice

arXiv:2609.06862v1 Announce Type: new Abstract: Superposition refers to neural networks representing more features than they have dimensions. It offers a possible explanation for polysemantic neurons...

By Dai Shi, Xiaoyu Li, Andi Han, Jos\'e Miguel Hern\'andez-Lobato

arXiv Statistics ML

Sep 10

High-probability guarantees for linear accessibility in feature superposition

arXiv:2609.09556v1 Announce Type: new Abstract: Neural networks can leverage feature superposition to encode more concepts than dimensions, but cross-feature interference constrains the linear access...

By Enrico Vompa

safety

arXiv Machine Learning

Jun 5

Expand Neurons, Not Parameters

arXiv:2510. 04500v3 Announce Type: replace Abstract: This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance.

By Linghao Kong, Inimai Subramanian, Yonadav Shavit, Micah Adler, Dan Alistarh, Nir Shavit

rag safety

arXiv Machine Learning

Aug 10

Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data

arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.

By Ege Erdogan, Ana Lucic

safety

arXiv Machine Learning

Jun 24

Similarity of Neural Network Representations in Superposition

arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.

By Sunny Liu, Habon Issa, Andr\'e Longon, Liv Gorton, Meenakshi Khosla, Alex Williams, David Klindt

llms safety

One email a morning, machine-written

Related stories

Compressed Computation under $L^4$ Loss is likely Computation in Superposition

Feature Superposition in Neural Networks: From Theory to Practice

High-probability guarantees for linear accessibility in feature superposition

Expand Neurons, Not Parameters

Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data

Similarity of Neural Network Representations in Superposition