arXiv:2606. 12747v1 Announce Type: new Abstract: Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs.
By Andy Wang, Parv Mahajan, David Demitri Africa, Alexandra Souly, Jordan Taylor, Robert Kirk
The paper introduces activation probes—directions in a language model’s residual stream—that can both detect and steer safety‑relevant behaviors such as deception, high‑stakes awareness, and sycophancy. By evaluating different estimators (difference‑of‑means vs. logistic regression) and token‑pooling schemes (last, mean, learned attention) on Llama‑3.1‑8B‑Instruct, the authors find that logistic regression excels at detection while difference‑of‑means excels at steering, except for sycophancy where the pattern reverses. The study highlights that the optimal pooling method, layer choice, and steering metric vary by behavior and data, and provides exported probes that enable an interactive interface for scoring and steering text.
By Nimun Bajwa, Massimo Poesio
arXiv:2606. 15420v1 Announce Type: cross Abstract: A constitution tells a language model what to value, but little tells us whether it does.
By Tong Che, Rui Wu
arXiv:2607. 10251v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly used in decision support, it is important to understand whether their choices under uncertainty exhibit stable and interpretable behavioural regularities.
By Xuankun Rong, Wenke Huang, Bo Du, Dacheng Tao, Mang Ye
arXiv:2611.03374v1 Announce Type: cross
Abstract: Recent language model (LM) safety failures have largely been studied as disparate phenomena. In agentic misalignment, a model blackmails an executive...
By Brandon Jaipersaud, Elliot Creager
The paper introduces RiskCueBench, a benchmark designed to separate intent recovery from safety behavior in large language models. It evaluates 25 model groups on 640 prompts covering 160 latent intents across eight risk categories, revealing that higher contextual specificity can reduce refusal rates and safety scores even after intent recovery. The study highlights that safety behavior may be miscalibrated to prompt surface form, underscoring the need for evaluations that condition on semantic recoverability.
By Yuwei Wang, Xingwang Liang, Yu Xie, Haotian Chi, Haijun Geng
The paper evaluates safety monitors by measuring recall only on prompts that the target model actually answers, rather than on all harmful prompts. Across several guard systems, recall at a 1% false‑positive rate drops sharply when focusing on answered prompts, with monitors catching refused requests 1.1–6.4 times more often than answered ones. Rewriting prompts to be less explicit dramatically increases compliance and reveals that many harmful requests slip past monitors, especially when phrasing is softened. Fine‑tuning guards on these rewritten prompts improves recall from 0.24 to 0.89 on answered requests and generalizes to unseen benchmarks.
By Sripad Karne
The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), an evidence‑grounded framework that improves safety for vision‑language models (VLMs). CNRF builds context‑matched benign‑harmful counterfactual pairs, uses their representation differences to define risk directions, and separates neighborhood support from risk estimation. This approach allows selective intervention rather than blanket rejection, achieving high macro defense success rates across multiple VLMs while preserving benign‑task utility.
By Tianshi Wang, Jingsong Wang, Fengling Li, Jingjing Li, Xin Li, Lei Zhu
Persona prompting is widely used in LLM user simulation, yet it assumes that specifying a target user attribute will shift only that attribute while leaving others unchanged. Across multiple models, the study finds that changing a target attribute also alters unspecified non‑target choices, creating a "neutrality gap" that persists even when no preference is declared. The authors show that blocking query‑time attention reduces non‑target drift, but this also weakens target efficacy, highlighting a central challenge in achieving selective control.
By Jiashen Ren, Wenlin Zhang, Bohan Zhang, Xiaopeng Li, Zichuan Fu, Wanyu Wang, Junyi Li, Xiangyu Zhao
arXiv:2610.16761v1 Announce Type: new
Abstract: LLM-based information systems need safety guardrails. Most guardrails classify the prompt alone. If the prompt looks harmful, the system rejects it bef...
By Ram Vikas Mishra, Shashank Aggarwal, Jahnavi Kumar, Amit Awekar
arXiv:2607. 15218v1 Announce Type: new Abstract: Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world.
By Weimeng Wang, Ziqiang Wang, Zihang Zhan, Chuanpu Fu, Qi Li, Ke Xu
arXiv:2606. 05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk categories, and/or explanatory rationales about potential policy violations.
By Yuhao Sun, Jiacheng Zhang, Shaanan Cohney, Zhexin Zhang, Feng Liu, Xingliang Yuan