arXiv Machine Learning
Oct 23

Probing and Steering for Safety-Relevant Behaviours in Large Language Models

The paper introduces activation probes—directions in a language model’s residual stream—that can both detect and steer safety‑relevant behaviors such as deception, high‑stakes awareness, and sycophancy. By evaluating different estimators (difference‑of‑means vs. logistic regression) and token‑pooling schemes (last, mean, learned attention) on Llama‑3.1‑8B‑Instruct, the authors find that logistic regression excels at detection while difference‑of‑means excels at steering, except for sycophancy where the pattern reverses. The study highlights that the optimal pooling method, layer choice, and steering metric vary by behavior and data, and provides exported probes that enable an interactive interface for scoring and steering text.

By Nimun Bajwa, Massimo Poesio
arXiv AI
Oct 19

Recoverability-Controlled Safety Miscalibration in Large Language Models

The paper introduces RiskCueBench, a benchmark designed to separate intent recovery from safety behavior in large language models. It evaluates 25 model groups on 640 prompts covering 160 latent intents across eight risk categories, revealing that higher contextual specificity can reduce refusal rates and safety scores even after intent recovery. The study highlights that safety behavior may be miscalibrated to prompt surface form, underscoring the need for evaluations that condition on semantic recoverability.

By Yuwei Wang, Xingwang Liang, Yu Xie, Haotian Chi, Haijun Geng