arXiv AI
The Sharp Directions Are Against You: Curvature Analysis of Activation Steering
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 10
Steering Interference Reflects the Model's Defaults, Not the Behavior Directions
arXiv:2609.06951v1 Announce Type: cross Abstract: Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activa...
arXiv Machine Learning
Jun 9
Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior
arXiv:2606. 08454v1 Announce Type: new Abstract: Activation steering provides a lightweight inference-time mechanism for controlling large language models (LLMs) by modifying their internal activation vectors toward desired behaviors.
arXiv Machine Learning
Aug 27
Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
The paper investigates whether fine‑tuning a language model erases previously embedded activation steering interventions that suppress refusals and encourage brevity. Across five instruction‑tuned models (3B–14B) subjected to non‑adversarial supervised fine‑tuning (SFT) and reinforcement learning from human feedback (RLHF), the authors find that the steering’s behavioural effect degrades when the fine‑tuning objective conflicts with the targeted behaviour, yet the underlying weight edits remain largely unchanged. Mechanistically, the steering vectors survive with minimal alteration, but functionally the steering is vulnerable and must be re‑validated after downstream training.
arXiv AI
Jul 17
Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control
arXiv:2607. 14943v1 Announce Type: cross Abstract: World Action Models (WAMs) enable semantically- and physically-informed control but are brittle under distribution shift.
arXiv Machine Learning
Sep 22
Look Before You Steer: Geometry Predicts SAE Feature Steerability
The paper investigates whether properties of a Self‑Attention Encoder (SAE) that can be computed before any forward pass can predict how difficult it is to steer individual SAE features. It finds that decoder‑space geometry—specifically neighbor density and maximum cosine similarity to nearby decoder directions—partially predicts feature steerability, with significant correlations across multiple model scales, widths, and architectures. The relationship holds up to a certain layer depth, suggesting a boundary where steering becomes too costly.
arXiv Machine Learning
Sep 10
When Does Activation Steering Change What a Model Computes From?
The paper investigates whether modifying a model’s internal activations—through activation steering—actually changes the computational state used in subsequent processing or merely biases the computation toward a desired output. In a controlled state‑tracking task, editing a trace‑supervised register causes the model to apply the next operation to the edited state, confirming that the edit changes the state. However, in two large‑language‑model settings (Qwen and Llama), mean activation steering does not replicate the natural internal configuration used during task execution; steering vectors are far larger than typical natural changes and achieve only a fraction of the effect of full‑layer patching. Thus, a successful steering intervention does not necessarily reproduce the natural target activation at the intervention layer, and should be interpreted as a state change only when later computation actually uses the edited value in the intended semantic way.