arXiv Machine Learning July 24, 2026 By Seonglae Cho, Zekun Wu, Kleyton Da Costa, Rishi Kalra, Ilham Wicaksono, Adriano Koshiyama

Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects

Read the original on arXiv Machine Learning →

arXiv:2607. 20596v1 Announce Type: new Abstract: Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet whether a feature's causal role is stable across SAE families remains untested.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 9

Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects

arXiv:2606. 08365v1 Announce Type: cross Abstract: Sparse autoencoder (SAE) features are increasingly used to steer language models, but feature steering is rarely clean: the same intervention can behave inconsistently across contexts and perturb unrelated features.

By Evan Duan

llms

arXiv Machine Learning

Jun 18

From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability

arXiv:2606. 18383v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable features from language models (LMs), yet a central question remains: when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM We study this through a post-hoc generalization framework that certifies the LM via a sparse proxy, obtained by replacing a native hidden activation with its pretrained SAE reconstruction.

By Dibyanayan Bandyopadhyay, Asif Ekbal

llms efficiency safety

arXiv AI

Jun 3

How Quantization Changes Interpretable Features: A Sparse Autoencoder Analysis of Language Models

arXiv:2606. 03002v1 Announce Type: cross Abstract: Quantization is a standard path to deploying large language models, and a quantized model is typically judged acceptable when its perplexity or downstream accuracy stays close to the full-precision original.

By Evan Duan

llms efficiency safety

arXiv Machine Learning

Aug 4

Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations

arXiv:2605. 28149v2 Announce Type: replace Abstract: Sparse Autoencoders (SAEs) extract interpretable features from Large Language Model activations, but standard variants enforce non-negative latents, so a bidirectional semantic axis (e.

By Bartosz Wieciech, Zmnako Awrahman, Marcin Czelej, Victor Hugo Jaramillo Velasquez, Wioletta Stobieniecka

llms efficiency safety

arXiv AI

Jul 14

When Are Sparse Feature Interventions Actually Localized? Matched Evaluation for SAE-Based Safety Control

arXiv:2607. 10226v1 Announce Type: new Abstract: We evaluate when sparse autoencoder (SAE) features act as localized control handles for safety-relevant behavior.

By Daming Luo

benchmarks safety

arXiv AI

Jul 28

Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects

arXiv:2607. 24645v1 Announce Type: cross Abstract: The wide-scale use of sparse autoencoders (SAEs) as interpretability tools is limited by inconsistent links between SAE features and model behavior.

By Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty, Iryna Gurevych, Subhabrata Dutta

safety

Related stories

Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects

From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability

How Quantization Changes Interpretable Features: A Sparse Autoencoder Analysis of Language Models

Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations

When Are Sparse Feature Interventions Actually Localized? Matched Evaluation for SAE-Based Safety Control

Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects