arXiv AI
Jun 30
A Gravitational Interpretation of Fine-Tuning Reversion
arXiv:2606. 28525v1 Announce Type: cross Abstract: Fine-tuning on harmless data can partially undo behaviors acquired earlier in training.
Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
The paper investigates whether fine‑tuning a language model erases previously embedded activation steering interventions that suppress refusals and encourage brevity. Across five instruction‑tuned models (3B–14B) subjected to non‑adversarial supervised fine‑tuning (SFT) and reinforcement learning from human feedback (RLHF), the authors find that the steering’s behavioural effect degrades when the fine‑tuning objective conflicts with the targeted behaviour, yet the underlying weight edits remain largely unchanged. Mechanistically, the steering vectors survive with minimal alteration, but functionally the steering is vulnerable and must be re‑validated after downstream training.
Related stories
arXiv AI
Oct 15
The Sharp Directions Are Against You: Curvature Analysis of Activation Steering
arXiv:2610.16084v1 Announce Type: new Abstract: Activation steering adds a vector to a language model's internal representations at inference time. It is a lightweight alternative to fine-tuning for...
arXiv Machine Learning
Jul 15
Inference-Time Machine Unlearning via Gated Activation Redirection
arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.
arXiv Machine Learning
Sep 10
When Does Activation Steering Change What a Model Computes From?
The paper investigates whether modifying a model’s internal activations—through activation steering—actually changes the computational state used in subsequent processing or merely biases the computation toward a desired output. In a controlled state‑tracking task, editing a trace‑supervised register causes the model to apply the next operation to the edited state, confirming that the edit changes the state. However, in two large‑language‑model settings (Qwen and Llama), mean activation steering does not replicate the natural internal configuration used during task execution; steering vectors are far larger than typical natural changes and achieve only a fraction of the effect of full‑layer patching. Thus, a successful steering intervention does not necessarily reproduce the natural target activation at the intervention layer, and should be interpreted as a state change only when later computation actually uses the edited value in the intended semantic way.
arXiv Machine Learning
Jun 19
Leverage Is Not Reach: A Control-Window Law for Single-Neuron Steering in Language Models
arXiv:2606. 19831v1 Announce Type: cross Abstract: Aligned language models gate behaviors such as refusal and language routing through sparse feed forward neurons, yet no theory predicts when a single neuron intervention controls a behavior coherently rather than collapsing the output.
arXiv AI
Aug 10
Divergent Response Modes in Frontier Language Models Under Steering Pressure
arXiv:2608. 06578v1 Announce Type: new Abstract: Frontier language models are trained using distinct data, objectives, and safety pipelines.
arXiv Machine Learning
Aug 11
Early Data Exposure Improves Robustness to Subsequent Fine-Tuning
arXiv:2605. 12705v2 Announce Type: replace Abstract: How can we train models whose post-trained capabilities survive subsequent fine-tuning?
arXiv AI
Sep 1
Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities
arXiv:2608.29458v1 Announce Type: cross Abstract: Sandbagging, in which a model deliberately underperforms on an evaluation despite retaining the underlying capability, threatens the safety evaluatio...
arXiv Machine Learning
Aug 11
Safety Cost of Steering Vectors Is Separable and Reducible
arXiv:2608. 08383v1 Announce Type: cross Abstract: Steering vectors are a lightweight tool for controlling LLM behavior.
arXiv AI
Jun 9
Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects
arXiv:2606. 08365v1 Announce Type: cross Abstract: Sparse autoencoder (SAE) features are increasingly used to steer language models, but feature steering is rarely clean: the same intervention can behave inconsistently across contexts and perturb unrelated features.
arXiv AI
Jul 29
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.
arXiv Computation and Language
Sep 10
Active Adaptation, Not Static Defense: Temporal Dynamics of Preventative Steering in Adversarial Fine-Tuning
arXiv:2609.10142v1 Announce Type: new Abstract: Large language models remain fragile against malicious fine-tuning, motivating training-time defenses against harmful persona drift. Preventative Steer...