arXiv:2606. 18383v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable features from language models (LMs), yet a central question remains: when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM We study this through a post-hoc generalization framework that certifies the LM via a sparse proxy, obtained by replacing a native hidden activation with its pretrained SAE reconstruction.
By Dibyanayan Bandyopadhyay, Asif Ekbal
arXiv:2602. 14687v2 Announce Type: replace-cross Abstract: Improving Sparse Autoencoders (SAEs) requires benchmarks that can precisely validate architectural innovations.
By David Chanin, Adri\`a Garriga-Alonso
arXiv:2506. 07691v2 Announce Type: replace-cross Abstract: Sparse Autoencoders (SAEs) are a cornerstone of mechanistic interpretability.
By Jiaming Li, Haoran Ye, Yukun Chen, Xinyue Li, Lei Zhang, Hamid Alinejad-Rokny, Jimmy Chih-Hsien Peng, Min Yang
arXiv:2606. 27321v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have become a leading tool for interpreting the representations of vision foundation models, decomposing their polysemantic activations into a larger set of sparse, more monosemantic features.
By Nathana\"el Jacquier, Maria Vakalopoulou, Mahdi S. Hosseini
arXiv:2605. 18629v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) are one of the main methods to interpret the inner workings of deep neural networks (DNNs), decomposing activations into higher-dimensional features.
By Micha{\l} Brzozowski, Neo Christopher Chung
arXiv:2606. 14990v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are standard tools for mechanistic interpretability, but current SAE families are constrained by fixed encoder nonlinearities such as ReLU, JumpReLU, and TopK.
By Naiyu Yin, Yue Yu
The paper investigates Sparse Autoencoders (SAEs) by applying dictionary learning identifiability results to derive constraints on optimal dictionary learning features. It shows that optimal features cannot activate simultaneously, explaining SAE oddities such as hierarchical splitting, absorption, dense antipodal features, and infinite feature splitting as consequences of the dictionary learning objective. The derived constraints also serve as diagnostics, revealing that real SAEs perform well on training data but fail on increasingly distant test data.
By William Dorrell
arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.
By Ege Erdogan, Ana Lucic
arXiv:2609.06557v1 Announce Type: new
Abstract: Large language models (LLMs) are often considered fragile under aggressive sparsification, and maintaining reliable performance typically requires stic...
By Hyeondo Jang, Kwanhee Lee, Dongyeop Lee, Namhoon Lee
The paper introduces Dynamic DAE Guardrails (DSG), a method that uses Dynamic Sparse Autoencoders to perform precision unlearning in large language models. DSG leverages principled feature selection and a dynamic classifier to target activation-based unlearning, outperforming existing gradient‑based methods in terms of computational efficiency, stability, sequential unlearning, resistance to relearning attacks, data efficiency, and interpretability.
By Aashiq Muhamed, Jacopo Bonato, Mona Diab, Virginia Smith
arXiv:2606. 02385v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) have found success parsing neural representations into interpretable concepts, providing a basis for understanding and control.
By William Dorrell
arXiv:2607. 20596v1 Announce Type: new Abstract: Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet whether a feature's causal role is stable across SAE families remains untested.
By Seonglae Cho, Zekun Wu, Kleyton Da Costa, Rishi Kalra, Ilham Wicaksono, Adriano Koshiyama