arXiv:2506. 23845v2 Announce Type: replace-cross Abstract: While sparse autoencoders (SAEs) have generated significant excitement, a series of negative results have added to skepticism about their usefulness.
arXiv:2610.20139v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) have emerged as a promising approach to mechanistic interpretability by decomposing language-model activations into sparse,...
arXiv:2609.01322v1 Announce Type: cross Abstract: In many settings, studying causal questions based on text data requires adjusting for confounding information within texts. Yet there is a tradeoff i...
The paper introduces CAST, a concept-guided artifact suppression tuning framework that uses sparse autoencoders to identify and suppress note-specific artifacts in clinical language models. CAST labels latent features with an LLM-assisted pipeline and ICD‑10 constraints, then fine‑tunes the model while providing post‑hoc per‑concept attributions for auditability. In experiments on MIMIC‑IV discharge‑note mortality prediction, CAST outperforms standard fine‑tuned encoders and competes with strong LLM baselines while offering a feature‑level audit trail of clinical concepts and suppressed artifacts.
The paper introduces a method to transform opaque fine‑tuned neural AI‑text detectors into interpretable classifiers by training sparse autoencoders on the detector’s residual stream. Each autoencoder feature is labeled with a human‑readable concept, and only features meeting a fidelity threshold are kept. A lightweight classifier built on these interpretable features outperforms both neural and stylometric detectors, especially on out‑of‑distribution texts.
arXiv:2606. 18383v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable features from language models (LMs), yet a central question remains: when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM We study this through a post-hoc generalization framework that certifies the LM via a sparse proxy, obtained by replacing a native hidden activation with its pretrained SAE reconstruction.
arXiv:2508.21285v4 Announce Type: replace-cross Abstract: We use sparse autoencoders (SAEs) to "brain scan" large language models, mapping internal activations to interpretable concepts and enabling...
In many settings, studying causal questions based on text data requires adjusting for confounding information within texts. Yet there is a tradeoff in constructing text representations for adjustment:...
arXiv:2512. 10092v2 Announce Type: replace Abstract: Analyzing large-scale text corpora is a core challenge in machine learning, crucial for tasks like identifying undesirable model behaviors or biases in training data.
The paper introduces Span-Centric Learning (SCL), a method that decouples coding knowledge acquisition from its application in full clinical documents. SCL first trains on short text spans to learn coding knowledge, then fine‑tunes on a small set of annotated documents, enabling an 8B LLM to outperform state‑of‑the‑art methods with only 248 full documents. The approach also yields explicit, human‑aligned supporting evidence for each predicted ICD code and can learn code knowledge from spans even when the codes are absent from the full documents.
arXiv:2606.22994v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) have become an important tool for unsupervised concept discovery in large models. To make the resulting feature spaces m...
Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale.
The paper introduces ICON decomposition, a method for auditing neural networks by decomposing the variance of intermediate representations into contributions from Independent Canonical Concepts (ICON). Unlike traditional decodability metrics, ICON isolates each concept’s influence while conditioning on all other concepts and the outcome, providing a clearer view of which concepts truly drive predictions. Experiments on simulated data and real-world skin‑cancer and neuroimaging tasks show ICON outperforms seven baseline methods and successfully distinguishes genuine shortcuts from merely correlated features.