arXiv:2607. 19364v1 Announce Type: new Abstract: Activation steering offers a lightweight alternative to fine-tuning for behavioral control of large language models, but SAE-based steering methods often rely on learned steering objectives or single-criterion feature selection.
arXiv:2607. 19364v2 Announce Type: replace Abstract: Activation steering adds a residual-stream direction at inference time, providing lightweight behavioral control without fine-tuning.
arXiv:2606. 11599v1 Announce Type: cross Abstract: Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration.
arXiv:2608.28806v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) disentangle model activations into interpretable features and are widely used for steering large language models. Most exist...
The paper introduces DSPA, a dynamic sparse autoencoder (SAE) steering technique that aligns language model outputs with user preferences during inference, avoiding costly weight updates. DSPA constructs a conditional-difference map from preference triples to adjust token-active latents, improving MT‑Bench scores and matching AlpacaEval performance on models like Gemma‑2 and Qwen3 while preserving accuracy. It demonstrates robustness with limited preference data, outperforms the two‑stage RAHF‑SCIT pipeline in FLOPs, and reveals that preference directions are largely driven by discourse and stylistic cues.
arXiv:2608. 05164v1 Announce Type: cross Abstract: Independently trained large language models may develop shared internal representations of semantic concepts despite architectural differences -- but whether this geometric similarity has functional consequences for cross-model behavioural control remains untested.
arXiv:2606. 08365v1 Announce Type: cross Abstract: Sparse autoencoder (SAE) features are increasingly used to steer language models, but feature steering is rarely clean: the same intervention can behave inconsistently across contexts and perturb unrelated features.
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.
arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.
The paper investigates whether properties of a Self‑Attention Encoder (SAE) that can be computed before any forward pass can predict how difficult it is to steer individual SAE features. It finds that decoder‑space geometry—specifically neighbor density and maximum cosine similarity to nearby decoder directions—partially predicts feature steerability, with significant correlations across multiple model scales, widths, and architectures. The relationship holds up to a certain layer depth, suggesting a boundary where steering becomes too costly.
arXiv:2609.06289v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inferenc...
Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts.
arXiv:2605. 05983v2 Announce Type: replace Abstract: Recently, steering vectors (SVs) have emerged as an effective and lightweight approach to steer behaviors of large language models (LLMs), among which fine-tuned SVs are more effective than optimization-free ones.