arXiv AI July 3, 2026 By Rodrigo Mendoza-Smith

Expander Sparse Autoencoders: Parameter-Efficient Dictionaries for Mechanistic Interpretability

arXiv:2607. 01799v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) decompose internal activations of neural networks into sparse linear combinations of learned features by fitting an overcomplete dictionary $\mathbf{W}\in\mathbb{R}^{m\times n}$ with $m<n$, and inferring a sparse code $\mathbf{x}\in\mathbb{R}^n$ from $\mathbf{h}\approx\mathbf{W}\mathbf{x}$.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jun 26

Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders

arXiv:2606. 27321v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have become a leading tool for interpreting the representations of vision foundation models, decomposing their polysemantic activations into a larger set of sparse, more monosemantic features.

By Nathana\"el Jacquier, Maria Vakalopoulou, Mahdi S. Hosseini

efficiency

arXiv Machine Learning

Jun 5

Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability

arXiv:2606. 06333v1 Announce Type: new Abstract: Sparse Autoencoders (SAEs) are widely used for mechanistic interpretability in large language models, yet their formulation assigns each latent feature a single decoder direction, implicitly assuming features to be one-dimensional.

By Seyed Arshan Dalili, Mehrdad Mahdavi

llms efficiency safety

arXiv Machine Learning

Jun 2

How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations

arXiv:2606. 02385v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) have found success parsing neural representations into interpretable concepts, providing a basis for understanding and control.

By William Dorrell

arXiv Machine Learning

Jun 15

Decompose Sparsely Where You Should, Absorb Densely Where You Should No

arXiv:2606. 14040v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained to reconstruct the \textbf{entire} residual stream through a sparse dictionary, implicitly assuming that all activation content is amenable to sparse, monosemantic decomposition.

By Ruixuan Deng, Zehao Jin, Zekun Wang, Zihan Dong

efficiency benchmarks safety

arXiv Machine Learning

Aug 4

SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits

arXiv:2608. 00859v1 Announce Type: new Abstract: Kolmogorov--Arnold Networks (KANs) replace scalar edge weights with learnable univariate functions parameterized by multiple basis coefficients.

By Kazi Ahmed Asif Fuad, Lizhong Chen

efficiency

arXiv AI

Aug 12

Post-Hoc Sparse Coding of Latent Communication Between Vision-Language Model Agents

arXiv:2608. 10198v1 Announce Type: new Abstract: Latent-space communication allows heterogeneous vision-language model agents to exchange continuous representations without serializing visual and reasoning states into text.

By Di Wu, Xiaohui Zhu

llms agents multimodal benchmarks

Related stories

Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders

Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability

How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations

Decompose Sparsely Where You Should, Absorb Densely Where You Should No

SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits

Post-Hoc Sparse Coding of Latent Communication Between Vision-Language Model Agents