arXiv:2607. 20522v1 Announce Type: new Abstract: This paper tests whether holonomy concentrates on active sparse-autoencoder (SAE) feature planes in Gemma 2 2B, a concrete operationalization of the broader semantic-concentration prediction.
arXiv:2606. 08365v1 Announce Type: cross Abstract: Sparse autoencoder (SAE) features are increasingly used to steer language models, but feature steering is rarely clean: the same intervention can behave inconsistently across contexts and perturb unrelated features.
arXiv:2607. 10226v1 Announce Type: new Abstract: We evaluate when sparse autoencoder (SAE) features act as localized control handles for safety-relevant behavior.
arXiv:2607. 20596v1 Announce Type: new Abstract: Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet whether a feature's causal role is stable across SAE families remains untested.
Linear probes can decode safety‑relevant concepts such as truthfulness from language‑model activations, but probe accuracy may reflect only decodability, not causal influence on model behavior. The authors show that probe weight geometry alone cannot identify the features the model actually uses, because geometrically aligned features need not be causally relevant. They introduce a sparse‑autoencoder (SAE) decomposition that ranks features by probe alignment and gradient sensitivity, and demonstrate that ablating shared, probe‑only, and random feature sets reveals a sharp dissociation: shared features drive model output changes far more than probe‑only or random features, confirming that causal relevance requires intervention beyond weight geometry.
arXiv:2609. 12591v1 Announce Type: new Abstract: Foundation models are increasingly adapted through fine-tuning, model editing, and alignment procedures while retaining previously acquired capabilities.
arXiv:2606. 18322v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) decompose residual-stream activations into interpretable features.
The paper reports three pre‑registered studies on 2D Navier‑Stokes flow that examine whether self‑supervised models organize their latent spaces around physical fluid quantities. In Study 1, a joint‑embedding predictive architecture (JEPA) fails to outperform an untrained network in capturing fluid dynamics, while a JEPA trained on non‑physical dynamics and a static invariance model perform equally well or better. Study 2 shows that plain autoencoders make enstrophy prominent in their slowest latent directions, and adding VICReg reduces this prominence but improves linear decodability. Study 3 constructs steerable latent directions (“dials”) via routing; a loudness dial is found in all eight trained models and a shape dial in three of five seeds, though neither control shows a shape dial. Across all studies, the content of a latent and what can be controlled through it diverge, and exploratory experiments reveal that supervised spectral‑shape directions do not affect the output when steered.
arXiv:2605. 03160v2 Announce Type: replace Abstract: The standard protocol for interpreting sparse-autoencoder (SAE) features labels each feature from its top-activating contexts and validates the label by steering that single feature at a typical magnitude.
arXiv:2605. 28149v2 Announce Type: replace Abstract: Sparse Autoencoders (SAEs) extract interpretable features from Large Language Model activations, but standard variants enforce non-negative latents, so a bidirectional semantic axis (e.
The paper investigates whether public medical vision‑language benchmarks contain data that may have leaked into the training sets of open VLMs. Using four contamination detectors, the authors find that a significant portion of images in SLAKE‑En have close neighbours in PMC‑OA‑beta, indicating source overlap rather than exact memorization. On the text side, only the exchangeability detector consistently flags potential contamination, while the other methods produce false positives and are unreliable without external baselines.
arXiv:2607. 24645v1 Announce Type: cross Abstract: The wide-scale use of sparse autoencoders (SAEs) as interpretability tools is limited by inconsistent links between SAE features and model behavior.
arXiv:2607. 12166v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features, and evaluation relies predominantly on correlational recovery metrics -- cosine similarity between ground-truth directions and decoder atoms.