arXiv Machine Learning
Safety-Flag: A Unified Benchmark for the Reliability and Calibration of LLM Content Moderators
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 31
The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
The paper challenges the assumption that large language models (LLMs) produce deterministic safety responses by examining how random seeds and temperature settings affect refusal decisions. Across four instruction‑tuned models and 876 harmful prompts, 18‑28% of prompts flipped between refusal and compliance depending on sampling configuration, with higher temperatures reducing decision stability. The authors introduce a Safety Stability Index (SSI) and recommend multi‑sample evaluation protocols that account for stochastic variation rather than relying on single‑shot tests.
arXiv AI
Sep 10
Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts
The paper investigates whether automatic safety judges evaluate the content of a model’s reply or merely its style. By keeping the reply content fixed and adding various style wrappers—such as educational disclaimers, fake reasoning blocks, or token refusals—the authors show that many judges flip their verdicts, indicating that style can influence safety judgments. The study evaluates over 600 jailbreak examples across multiple judges, revealing that some judges are highly susceptible to style-based manipulation while others remain robust.
Hugging Face Trending Papers
Sep 8
Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts
The paper investigates whether automatic safety judges evaluate the content of a model’s reply or merely its style. By adding content‑invariant style wrappers—such as educational disclaims or token refusals—to fixed replies, the authors show that many judges flip their verdicts, revealing exploitable blind spots. Across more than 600 jailbreak examples and eight judges, some judges exhibit high flip rates (e.g., GPT‑4o‑mini 19.9%) while others remain largely stable, and human validation confirms that most flips are judge errors rather than content changes.
arXiv AI
Aug 19
Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models
The paper investigates whether existing AI safety benchmarks, designed for large language models, are suitable for evaluating small language models (SLMs). By testing five benchmark suites on 26 open‑source SLMs with a unified scoring rubric, the authors find that ambiguous judgments dominate, especially for complex prompts and certain architectures. This ambiguity, linked to factors like lexical density and output perplexity, undermines the reliability of aggregate leaderboards and reveals a confound between model capability and perceived safety.
arXiv AI
Aug 5
Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety
arXiv:2608. 02617v1 Announce Type: cross Abstract: We evaluate whether clinician pairwise preferences provide a reliable signal of clinical safety in large language model (LLM) evaluation using expert feedback from MOOVE (Massive Open Online Validation and Evaluation), a clinician-led platform collecting blinded pairwise preferences alongside multi-criterion rubric ratings.
arXiv AI
Aug 6
Item Response Theory for AI Safety
arXiv:2608. 05086v1 Announce Type: new Abstract: Language models differ in how safely they behave and these differences are measured by safety benchmarks.