IDEA is a training‑free, input‑dependent steering method for large language models that matches activations to cluster‑specific directions aligned with a target concept. It clusters positive and negative activation supports per attention head, solves an optimal‑matching problem to create a pool of cluster‑conditional directions, and selects the best match for each input at inference time. This approach preserves the input’s original representation while improving the truth × info rate on TruthfulQA by an average of 9.9% (up to 23.5%) over input‑independent baselines.
By Zheng Wang, Muchen Li, Renjie Liao, Yan Leng
The paper introduces genesis steering, a method that derives a steering vector from contrastive examples in a model’s own fine‑tuning data, and proposes using membership inference to build proxy datasets when such data are unavailable. Two theoretical results are presented on optimal displacement to the decision boundary and reduced proxy‑estimation bias. Experiments on T"ulu‑3‑8B and Llama‑3.1‑8B‑Instruct show that safety data from T"ulu‑3‑8B’s training mixture produce stronger steering than external datasets, and that membership‑based proxy construction can achieve competitive performance.
By Tora Bodin, Eric Aubinais, Ilaria Torre, Chih-Hong Cheng
arXiv:2602.09870v3 Announce Type: replace
Abstract: Steering methods influence Large Language Model behavior by identifying semantic directions in hidden representations, but are typically realized t...
By Chung-En Sun, Ge Yan, Zimo Wang, Tsui-Wei Weng
arXiv:2610.14025v1 Announce Type: cross
Abstract: Activation steering controls large language model behavior by adding a direction to hidden activations during inference. For steering to work well, t...
By Alexandros Xenos, Manos Plitsis, Niki Foteinopoulou, Yannis Panagakis, Ioannis Patras
arXiv:2606. 15092v1 Announce Type: new Abstract: Activation steering has emerged as a key methodology for controlling the behavior of large language models (LLMs).
By Minh-Hieu Pham, Bach Do, Laziz Abdullaev, Tan Minh Nguyen, Khoat Than
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal