arXiv Machine Learning June 3, 2026 By Micha{\l} Brzozowski, Neo Christopher Chung

Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)

Read the original on arXiv Machine Learning →

arXiv:2605. 18629v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) are one of the main methods to interpret the inner workings of deep neural networks (DNNs), decomposing activations into higher-dimensional features.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 29

VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring

arXiv:2606. 27941v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) provide useful decompositions of Transformer residual streams, but their learned features are usually named post hoc rather than directly connected to the Transformer's token vocabulary.

By Kairui Zhang, Ziwen Yu, Zahraa S. Abdallah, Martha Lewis

llms rag safety

arXiv AI

Jul 14

SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data

arXiv:2602. 14687v2 Announce Type: replace-cross Abstract: Improving Sparse Autoencoders (SAEs) requires benchmarks that can precisely validate architectural innovations.

By David Chanin, Adri\`a Garriga-Alonso

llms benchmarks

arXiv AI

Jul 8

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

arXiv:2508. 16560v4 Announce Type: replace-cross Abstract: Sparse Autoencoders (SAEs) extract features from LLM internal activations, meant to correspond to interpretable concepts.

By David Chanin, Adri\`a Garriga-Alonso

llms efficiency

arXiv Machine Learning

Jun 4

BBOmix: A Tabular Benchmark for Hyperparameter Optimization of Unsupervised Biological Representation Learning

arXiv:2606. 05139v1 Announce Type: new Abstract: The rapid advancement of high-throughput sequencing has led to large, high-dimensional omics datasets.

By Luca Thale-Bombien, Jan Ewald, Ralf K\"onig, Aaron Klein

fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 24

Similarity of Neural Network Representations in Superposition

arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.

By Sunny Liu, Habon Issa, Andr\'e Longon, Liv Gorton, Meenakshi Khosla, Alex Williams, David Klindt

llms safety

arXiv AI

Jul 24

Scaling Interpretable Transformers with Parity Bottleneck Layers

arXiv:2607. 20652v1 Announce Type: cross Abstract: Language models are thought to exhibit the phenomenon of superposition, representing many more features than dimensions in their residual streams.

By Andrew Mack, Kraig Yuheng Tou, Mark Henry, Zhengxun Wu, Lauren Greenspan

llms efficiency safety

Related stories

VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring

SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

BBOmix: A Tabular Benchmark for Hyperparameter Optimization of Unsupervised Biological Representation Learning

Similarity of Neural Network Representations in Superposition

Scaling Interpretable Transformers with Parity Bottleneck Layers