Hugging Face Trending Papers

When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Making

arXiv Machine Learning
Oct 23

Probing and Steering for Safety-Relevant Behaviours in Large Language Models

The paper introduces activation probes—directions in a language model’s residual stream—that can both detect and steer safety‑relevant behaviors such as deception, high‑stakes awareness, and sycophancy. By evaluating different estimators (difference‑of‑means vs. logistic regression) and token‑pooling schemes (last, mean, learned attention) on Llama‑3.1‑8B‑Instruct, the authors find that logistic regression excels at detection while difference‑of‑means excels at steering, except for sycophancy where the pattern reverses. The study highlights that the optimal pooling method, layer choice, and steering metric vary by behavior and data, and provides exported probes that enable an interactive interface for scoring and steering text.

By Nimun Bajwa, Massimo Poesio
arXiv AI
Oct 19

Recoverability-Controlled Safety Miscalibration in Large Language Models

The paper introduces RiskCueBench, a benchmark designed to separate intent recovery from safety behavior in large language models. It evaluates 25 model groups on 640 prompts covering 160 latent intents across eight risk categories, revealing that higher contextual specificity can reduce refusal rates and safety scores even after intent recovery. The study highlights that safety behavior may be miscalibrated to prompt surface form, underscoring the need for evaluations that condition on semantic recoverability.

By Yuwei Wang, Xingwang Liang, Yu Xie, Haotian Chi, Haijun Geng
arXiv Computation and Language
Oct 1

Safety Monitors Mostly Catch What the Model Already Refuses

The paper evaluates safety monitors by measuring recall only on prompts that the target model actually answers, rather than on all harmful prompts. Across several guard systems, recall at a 1% false‑positive rate drops sharply when focusing on answered prompts, with monitors catching refused requests 1.1–6.4 times more often than answered ones. Rewriting prompts to be less explicit dramatically increases compliance and reveals that many harmful requests slip past monitors, especially when phrasing is softened. Fine‑tuning guards on these rewritten prompts improves recall from 0.24 to 0.89 on answered requests and generalizes to unseen benchmarks.

By Sripad Karne
arXiv AI
Oct 16

Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models

The paper introduces Counterfactual Neighborhood Risk Fields (CNRF), an evidence‑grounded framework that improves safety for vision‑language models (VLMs). CNRF builds context‑matched benign‑harmful counterfactual pairs, uses their representation differences to define risk directions, and separates neighborhood support from risk estimation. This approach allows selective intervention rather than blanket rejection, achieving high macro defense success rates across multiple VLMs while preserving benign‑task utility.

By Tianshi Wang, Jingsong Wang, Fengling Li, Jingjing Li, Xin Li, Lei Zhu
arXiv AI
3d ago

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation

Persona prompting is widely used in LLM user simulation, yet it assumes that specifying a target user attribute will shift only that attribute while leaving others unchanged. Across multiple models, the study finds that changing a target attribute also alters unspecified non‑target choices, creating a "neutrality gap" that persists even when no preference is declared. The authors show that blocking query‑time attention reduces non‑target drift, but this also weakens target efficacy, highlighting a central challenge in achieving selective control.

By Jiashen Ren, Wenlin Zhang, Bohan Zhang, Xiaopeng Li, Zichuan Fu, Wanyu Wang, Junyi Li, Xiangyu Zhao
arXiv AI
Jun 6

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

arXiv:2606. 05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk categories, and/or explanatory rationales about potential policy violations.

By Yuhao Sun, Jiacheng Zhang, Shaanan Cohney, Zhexin Zhang, Feng Liu, Xingliang Yuan