The paper investigates attribution graphs for sparse autoencoders (SAEs) by examining how zeroing a feature affects subsequent layer activations. It shows that in TopK SAEs, removing a feature often causes another feature to cross the activation cutoff, and these crossings account for most of the response’s energy. The authors demonstrate that a margin‑based graph, which adds first‑order changes to preactivations, achieves low error (≤0.033) while accurately capturing the impact of feature ablations.
arXiv:2609. 12591v1 Announce Type: new Abstract: Foundation models are increasingly adapted through fine-tuning, model editing, and alignment procedures while retaining previously acquired capabilities.
arXiv:2609.37604v1 Announce Type: new Abstract: Graph foundation models need a discrete token representation, but casting a graph as a generatable token sequence faces a structural obstacle: edges sp...
The paper introduces the Redistribution Sensitivity Score (RSS) to detect consistent changes in sparse feature activations of Sparse Autoencoders (SAEs) when prompts are transformed. RSS identifies large, directionally consistent feature responses and evaluates their impact on downstream readouts across 27 pretrained SAEs from eight language models. Experiments show that combining RSS support erosion with the clean margin predicts recurrent safety readout failures with high AUROC, demonstrating RSS’s utility in quantifying representation changes without conflating them with failure.
arXiv:2608. 13337v1 Announce Type: new Abstract: Sparse autoencoders are meant to name the things a language model computes, and the usual way to check that a latent matters is to switch it off and see what changes.
arXiv:2607. 20596v1 Announce Type: new Abstract: Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet whether a feature's causal role is stable across SAE families remains untested.
arXiv:2606. 12138v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are widely used to interpret neural network representations, but their utility depends on whether the learned features are reproducible across training runs.
The study measured the impact of a single training example on a GPT‑2 model by running 24 counterfactual experiments. 32 models were trained from scratch on OpenWebText, and at a specific training step a single batch row was replaced with a 194‑token passage under three conditions (fluent prose, fabricated subject, random characters) or left unchanged. Results showed that the passage was learned from one exposure and decayed, with measurable differences in cross‑entropy up to 50 steps after injection but no lasting effect at the final step.
arXiv:2611.07756v1 Announce Type: cross Abstract: Activation storage is often a bottleneck in sparse autoencoder (SAE) training. We study how many unique activations an SAE needs when its total train...
arXiv:2607. 12166v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features, and evaluation relies predominantly on correlational recovery metrics -- cosine similarity between ground-truth directions and decoder atoms.
arXiv:2611.08089v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models route each token to a few of many experts, yet what an individual expert is responsible for remains unsettled. Existing...
arXiv:2605. 28149v2 Announce Type: replace Abstract: Sparse Autoencoders (SAEs) extract interpretable features from Large Language Model activations, but standard variants enforce non-negative latents, so a bidirectional semantic axis (e.
arXiv:2606. 14990v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are standard tools for mechanistic interpretability, but current SAE families are constrained by fixed encoder nonlinearities such as ReLU, JumpReLU, and TopK.