arXiv Machine Learning
Feature Superposition in Neural Networks: From Theory to Practice
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 18
Effects of sparsity and superposition on loss in simple autoencoders
arXiv:2606. 18538v1 Announce Type: new Abstract: One of the major difficulties in the mechanistic interpretability of neural networks is the occurrence of polysemanticity, which suggests that each neuron is typically responsible for multiple different tasks, impeding a clean interpretation of their function.
arXiv Machine Learning
Jul 7
Compressed Computation under $L^4$ Loss is likely Computation in Superposition
arXiv:2607. 04800v1 Announce Type: new Abstract: Neural networks are thought to represent concepts as directions in their activation space, and superposition lets them encode more concepts than they have dimensions.
arXiv Machine Learning
Jun 24
Similarity of Neural Network Representations in Superposition
arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.
arXiv Machine Learning
Jun 24
Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
arXiv:2311. 02960v5 Announce Type: replace Abstract: Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data.
arXiv Machine Learning
Jun 5
Expand Neurons, Not Parameters
arXiv:2510. 04500v3 Announce Type: replace Abstract: This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance.
arXiv AI
Sep 10
Instance-wise Linearization of Neural Network for Model Interpretation
arXiv:2310.16295v2 Announce Type: replace-cross Abstract: Neural network have achieved remarkable successes in many scientific fields. However, the interpretability of the neural network model is sti...