The paper introduces a geometry-based metric, the discriminability index $d'$, to guide activation steering in large language models. By evaluating $d'$ across five behaviors and three models, the authors show it can reliably pinpoint effective layers for steering in multiple-choice question answering, reducing the need for exhaustive layer sweeps. While $d'$ also aids layer selection in open-ended multi-token generation, its ability to predict steering success on individual tokens is inconsistent, indicating limitations when extending from single-token to open-ended contexts.
arXiv:2606. 11599v1 Announce Type: cross Abstract: Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration.
Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts.
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
arXiv:2602.09870v3 Announce Type: replace Abstract: Steering methods influence Large Language Model behavior by identifying semantic directions in hidden representations, but are typically realized t...
arXiv:2610.14025v1 Announce Type: cross Abstract: Activation steering controls large language model behavior by adding a direction to hidden activations during inference. For steering to work well, t...
arXiv:2605. 05983v2 Announce Type: replace Abstract: Recently, steering vectors (SVs) have emerged as an effective and lightweight approach to steer behaviors of large language models (LLMs), among which fine-tuned SVs are more effective than optimization-free ones.
arXiv:2509. 03647v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly serve as automated evaluators, yet they suffer from "self-preference bias": a tendency to favor their own outputs over those of other models.
TopoSteer introduces a topology-based method for selecting the layer at which to apply activation steering in large language models. Instead of evaluating separability or alignment, it scores layers by the total H_0 persistence of the displacement cloud, measured as the length of its Euclidean minimum spanning tree. Experiments across several models and tasks show that this approach yields more effective steering without needing held-out validation.
arXiv:2609.15131v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) require substantial computation to process numerous visual tokens across all transformer layers. Most method...
arXiv:2604.24693v2 Announce Type: replace Abstract: Linear activation steering is a powerful approach for controlling and specializing large language models by adding linear representations of concep...
arXiv:2609.06289v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inferenc...
Kernelized Activation Steering (KAS) introduces a training‑free method for controlling attributes of generative models by lifting activation steering into a reproducing kernel Hilbert space. It formulates steering as an optimization problem using only kernel evaluations, producing an implicit, activation‑dependent steering score that adapts locally to the geometry of the activation space. Unlike standard Difference‑in‑Means, KAS generates a nonlinear steering field and recovers DiM as a special case with a linear kernel, achieving comparable or superior performance on tasks such as jailbreaking LLMs and image style control.