arXiv Computation and Language
Calibrating Lightweight Sparse Autoencoder Feature Steering
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 1
Enhancing SAE-based Steering via Neighbor Integrated Feature Selection
arXiv:2608.28806v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) disentangle model activations into interpretable features and are widely used for steering large language models. Most exist...
arXiv Machine Learning
Jun 30
Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
arXiv:2507. 23220v2 Announce Type: replace-cross Abstract: Traditional topic models are effective at uncovering latent themes in large text collections.
arXiv AI
Jun 30
C$^{2}$R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders
arXiv:2606. 30609v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) are widely used to interpret large language models by decomposing activations into sparse, human-understandable features, but scaling to large dictionaries exposes fundamental challenges.
arXiv Machine Learning
Jun 17
Stable and Steerable Sparse Autoencoders with Weight Regularization
arXiv:2603. 04198v2 Announce Type: replace-cross Abstract: Sparse autoencoders (SAEs) are widely used to extract human-interpretable features from neural network activations, but their learned features can vary substantially across random seeds and training choices.
arXiv Computation and Language
3d ago
Information-Theoretic Design of Sparse Autoencoder-Based Steering
The paper introduces an information‑theoretic framework for designing sparse autoencoder‑based steering interventions in large language models. It uses the Jensen–Shannon divergence between empirical feature‑firing distributions to quantify how much a state deviates from a target distribution and to construct steering vectors that guide hidden states toward desired firing patterns. Experiments demonstrate the method’s effectiveness in question‑mode generation, show structured downstream effects of local interventions, and apply the framework to safety‑related feature analysis.
arXiv Machine Learning
Sep 10
Disentangling Steering Vectors
arXiv:2609.07037v1 Announce Type: new Abstract: Activation steering has emerged as a lightweight, inference-time approach to control the behavior of Large Language Models (LLMs). However, traditional...