Probing and Steering for Safety-Relevant Behaviours in Large Language Models
Read the original on arXiv Machine Learning →The paper introduces activation probes—directions in a language model’s residual stream—that can both detect and steer safety‑relevant behaviors such as deception, high‑stakes awareness, and sycophancy. By evaluating different estimators (difference‑of‑means vs. logistic regression) and token‑pooling schemes (last, mean, learned attention) on Llama‑3.1‑8B‑Instruct, the authors find that logistic regression excels at detection while difference‑of‑means excels at steering, except for sycophancy where the pattern reverses. The study highlights that the optimal pooling method, layer choice, and steering metric vary by behavior and data, and provides exported probes that enable an interactive interface for scoring and steering text.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.