arXiv Machine Learning June 24, 2026 By Sunny Liu, Habon Issa, Andr\'e Longon, Liv Gorton, Meenakshi Khosla, Alex Williams, David Klindt

Similarity of Neural Network Representations in Superposition

Read the original on arXiv Machine Learning →

arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 10

Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data

arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.

By Ege Erdogan, Ana Lucic

safety

arXiv Machine Learning

Jun 3

Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)

arXiv:2605. 18629v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) are one of the main methods to interpret the inner workings of deep neural networks (DNNs), decomposing activations into higher-dimensional features.

By Micha{\l} Brzozowski, Neo Christopher Chung

efficiency benchmarks safety

arXiv AI

Jul 10

When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalities

arXiv:2607. 08605v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have emerged as a promising technique for mechanistic interpretability by learning a set of sparse latent features in large models, each of which encodes a distinct concept.

By Weiduo Liao, Yunqiao Yang, Ying Wei

llms efficiency multimodal safety

arXiv Machine Learning

Jun 18

Effects of sparsity and superposition on loss in simple autoencoders

arXiv:2606. 18538v1 Announce Type: new Abstract: One of the major difficulties in the mechanistic interpretability of neural networks is the occurrence of polysemanticity, which suggests that each neuron is typically responsible for multiple different tasks, impeding a clean interpretation of their function.

By Mriganka Basu Roy Chowdhury, Eric McLaughlin Weiner

efficiency safety

arXiv Machine Learning

Jul 7

Compressed Computation under $L^4$ Loss is likely Computation in Superposition

arXiv:2607. 04800v1 Announce Type: new Abstract: Neural networks are thought to represent concepts as directions in their activation space, and superposition lets them encode more concepts than they have dimensions.

By Francisco Ferreira da Silva, Stefan Heimersheim

arXiv Machine Learning

Jun 24

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination

arXiv:2311. 02960v5 Announce Type: replace Abstract: Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data.

By Peng Wang, Xiao Li, Can Yaras, Zhihui Zhu, Laura Balzano, Wei Hu, Qing Qu

fine-tuning

Related stories

Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data

Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)

When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalities

Effects of sparsity and superposition on loss in simple autoencoders

Compressed Computation under $L^4$ Loss is likely Computation in Superposition

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination