arXiv Computation and Language November 4, 2026 By Nishkal Hundia, Swastik Agrawal, Navita Goyal, Sarah Wiegreffe

Sweep Less, Steer More: Using Activation Steering Geometry for Reduced Hyperparameter Search

Read the original on arXiv Computation and Language →

The paper introduces a geometry-based metric, the discriminability index $d'$, to guide activation steering in large language models. By evaluating $d'$ across five behaviors and three models, the authors show it can reliably pinpoint effective layers for steering in multiple-choice question answering, reducing the need for exhaustive layer sweeps. While $d'$ also aids layer selection in open-ended multi-token generation, its ability to predict steering success on individual tokens is inconsistent, indicating limitations when extending from single-token to open-ended contexts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jun 11

When is Your LLM Steerable?

arXiv:2606. 11599v1 Announce Type: cross Abstract: Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration.

By Chenrui Fan, Yize Cheng, Ming Li, Soheil Feizi, Tianyi Zhou

llms

Hugging Face Trending Papers

Jun 10

When is Your LLM Steerable?

Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts.

llms

arXiv AI

Jul 13

GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs

arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.

By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

llms fine-tuning multimodal safety

arXiv Computation and Language

1d ago

Steer2Edit: From Activation Steering to Component-Level Editing

arXiv:2602.09870v3 Announce Type: replace Abstract: Steering methods influence Large Language Model behavior by identifying semantic directions in hidden representations, but are typically realized t...

By Chung-En Sun, Ge Yan, Zimo Wang, Tsui-Wei Weng

llms safety

arXiv AI

Oct 15

Contrastive Covariance Subspaces for Activation Steering

arXiv:2610.14025v1 Announce Type: cross Abstract: Activation steering controls large language model behavior by adding a direction to hidden activations during inference. For steering to work well, t...

By Alexandros Xenos, Manos Plitsis, Niki Foteinopoulou, Yannis Panagakis, Ioannis Patras

llms benchmarks

arXiv Machine Learning

Jun 15

Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions

arXiv:2605. 05983v2 Announce Type: replace Abstract: Recently, steering vectors (SVs) have emerged as an effective and lightweight approach to steer behaviors of large language models (LLMs), among which fine-tuned SVs are more effective than optimization-free ones.

By Yuntai Bao, Qinfeng Li, Xinyan Yu, Ge Su, Wenqi Zhang, Liu Yan, Haiqin Weng, Jianwei Yin, Xuhong Zhang

llms fine-tuning safety

One email a morning, machine-written

Related stories

When is Your LLM Steerable?

When is Your LLM Steerable?

GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs

Steer2Edit: From Activation Steering to Component-Level Editing

Contrastive Covariance Subspaces for Activation Steering

Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions