IDEA: training-free Input-Dependent stEEring via Activation cluster matching (IDEEA) is a method that steers large language models by injecting bias into selected activations at inference time, without requiring weight updates. Unlike existing training-free steering approaches that use a single, input-independent direction, IDEEA clusters positive and negative activation supports per attention head and solves an optimal-matching problem to create a set of cluster-conditional directions. At inference, IDEEA selects the direction that best matches the input’s activation, aligning the model toward a target concept while preserving the input’s original representation, and achieves a 9.9% average improvement in truth × info rate on TruthfulQA compared to the best input-independent baseline.
The paper introduces genesis steering, a method that derives a steering vector from contrastive examples in a model’s own fine‑tuning data, and proposes using membership inference to build proxy datasets when such data are unavailable. Two theoretical results are presented on optimal displacement to the decision boundary and reduced proxy‑estimation bias. Experiments on T"ulu‑3‑8B and Llama‑3.1‑8B‑Instruct show that safety data from T"ulu‑3‑8B’s training mixture produce stronger steering than external datasets, and that membership‑based proxy construction can achieve competitive performance.
By Tora Bodin, Eric Aubinais, Ilaria Torre, Chih-Hong Cheng
arXiv:2602.09870v3 Announce Type: replace
Abstract: Steering methods influence Large Language Model behavior by identifying semantic directions in hidden representations, but are typically realized t...
By Chung-En Sun, Ge Yan, Zimo Wang, Tsui-Wei Weng
arXiv:2610.14025v1 Announce Type: cross
Abstract: Activation steering controls large language model behavior by adding a direction to hidden activations during inference. For steering to work well, t...
By Alexandros Xenos, Manos Plitsis, Niki Foteinopoulou, Yannis Panagakis, Ioannis Patras
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal
arXiv:2605. 05983v2 Announce Type: replace Abstract: Recently, steering vectors (SVs) have emerged as an effective and lightweight approach to steer behaviors of large language models (LLMs), among which fine-tuned SVs are more effective than optimization-free ones.
By Yuntai Bao, Qinfeng Li, Xinyan Yu, Ge Su, Wenqi Zhang, Liu Yan, Haiqin Weng, Jianwei Yin, Xuhong Zhang