The paper introduces Where to Steer (W2S), a framework that adaptively selects which layer of a large language model to intervene on based on the input. It argues that optimal steering layers vary across inputs and demonstrates that W2S outperforms fixed-layer steering methods on several LLMs and alignment tasks, both in‑distribution and out‑of‑distribution. The study highlights input‑dependent layer selection as a complementary design dimension to steering vector extraction.
By Soham Gadgil, Chris Lin, Su-In Lee
arXiv:2608. 08829v1 Announce Type: cross Abstract: Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the injection layers globally per task.
By Muhammad Faishal Adly Nelwan, Alfan Farizki Wicaksono
The paper introduces a geometry-based metric, the discriminability index $d'$, to guide activation steering in large language models. By evaluating $d'$ across five behaviors and three models, the authors show it can reliably pinpoint effective layers for steering in multiple-choice question answering, reducing the need for exhaustive layer sweeps. While $d'$ also aids layer selection in open-ended multi-token generation, its ability to predict steering success on individual tokens is inconsistent, indicating limitations when extending from single-token to open-ended contexts.
By Nishkal Hundia, Swastik Agrawal, Navita Goyal, Sarah Wiegreffe
arXiv:2609.14151v1 Announce Type: cross
Abstract: Steering language models using a set of contrastive representations has been a canonical and computationally efficient method for controlling model b...
By Prajjwal Bhattarai, Tuka Alhanai
arXiv:2606. 08454v1 Announce Type: new Abstract: Activation steering provides a lightweight inference-time mechanism for controlling large language models (LLMs) by modifying their internal activation vectors toward desired behaviors.
By Tuc Nguyen, Thai Le
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal
The paper introduces genesis steering, a method that derives a steering vector from contrastive examples in a model’s own fine‑tuning data, and proposes using membership inference to build proxy datasets when such data are unavailable. Two theoretical results are presented on optimal displacement to the decision boundary and reduced proxy‑estimation bias. Experiments on T"ulu‑3‑8B and Llama‑3.1‑8B‑Instruct show that safety data from T"ulu‑3‑8B’s training mixture produce stronger steering than external datasets, and that membership‑based proxy construction can achieve competitive performance.
By Tora Bodin, Eric Aubinais, Ilaria Torre, Chih-Hong Cheng
IDEA is a training‑free, input‑dependent steering method for large language models that matches activations to cluster‑specific directions aligned with a target concept. It clusters positive and negative activation supports per attention head, solves an optimal‑matching problem to create a pool of cluster‑conditional directions, and selects the best match for each input at inference time. This approach preserves the input’s original representation while improving the truth × info rate on TruthfulQA by an average of 9.9% (up to 23.5%) over input‑independent baselines.
By Zheng Wang, Muchen Li, Renjie Liao, Yan Leng
arXiv:2606. 11599v1 Announce Type: cross Abstract: Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration.
By Chenrui Fan, Yize Cheng, Ming Li, Soheil Feizi, Tianyi Zhou
arXiv:2608. 05164v1 Announce Type: cross Abstract: Independently trained large language models may develop shared internal representations of semantic concepts despite architectural differences -- but whether this geometric similarity has functional consequences for cross-model behavioural control remains untested.
By Ayushi Agarwal
arXiv:2607. 16637v1 Announce Type: new Abstract: Full fine-tuning remains a strong way to adapt pretrained LLMs, but it updates all weights and can be expensive.
By Abdulkadir Erol, Yash Mahajan, Vepaul Hariprashad, Baha Rababah, Santu Karmaker, Cuneyt G. Akcora, Mubarak Shah
arXiv:2610.14025v1 Announce Type: cross
Abstract: Activation steering controls large language model behavior by adding a direction to hidden activations during inference. For steering to work well, t...
By Alexandros Xenos, Manos Plitsis, Niki Foteinopoulou, Yannis Panagakis, Ioannis Patras