← Back to all news
arXiv Machine Learning By Sahil Kadadekar

A Safe Prototype Is Not a Safety Direction: Reference Dependence and Prompt Confounds in Response-Safety Embeddings

Read the original on arXiv Machine Learning →

The paper investigates whether response safety can be measured by the cosine similarity between a response embedding and the mean embedding of known‑safe responses. Using four frozen encoders and prompt‑controlled datasets, the authors find that a simple prototype (mean safe embedding) performs poorly (ROC‑AUC 0.457‑0.545) while an explicit safe‑minus‑unsafe reference achieves higher scores (0.588‑0.738). The study shows that a class mean is merely a location, not a safety direction, and that a reference with sufficient unsafe mass is needed to orient safety judgments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jul 28

Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B

arXiv:2607. 22545v1 Announce Type: cross Abstract: Deploying large language models in financial-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm, a combination no existing open guardrail addresses in a single inference pass.

By Tejasvi C. Addagada
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Sep 10

Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts

The paper investigates whether automatic safety judges evaluate the content of a model’s reply or merely its style. By keeping the reply content fixed and adding various style wrappers—such as educational disclaimers, fake reasoning blocks, or token refusals—the authors show that many judges flip their verdicts, indicating that style can influence safety judgments. The study evaluates over 600 jailbreak examples across multiple judges, revealing that some judges are highly susceptible to style-based manipulation while others remain robust.

By Yongxi Zhou, Wenbo Ye, Yuanzhe Liu, Zihan Dong, Junwei Yao
llmsbenchmarkssafety
More like this →