← Back to all news
Hugging Face Trending Papers

Choosing Where and How to Moderate: End-to-End Trade-offs in Filter Placement and Response Rewriting

Read the original on Hugging Face Trending Papers →

Content-moderation classifiers are usually evaluated in isolation, but deployment requires choosing where to intervene and what follows a flag. We evaluate these choices using two end-to-end customer-outcome metrics rather than component accuracy: Usefulness, the fraction of turns with a shown, non-harmful, relevant response, and Harmful Exposure, the fraction with a shown harmful response.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

Related stories

arXiv Machine Learning
Aug 19

Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings

Reflex-Guard is a lightweight, locally‑run guardrail for large language models that uses jailbreak‑aware preprocessing, compact sentence‑transformer embeddings, and seven fast binary classifiers to filter unsafe prompts. It achieves 95.9 % recall on harmful prompts with an end‑to‑end latency of 37.6 ms, outperforming existing solutions such as Llama Guard 2 (255 ms) and SafeDecoding (723 ms). The system can detect all GCG suffix attacks and Base64‑encoded prompts at the default threshold, while DrAttack structured prompts require a lower threshold of 0.03 for optimal detection, and it attains a Reflex Efficiency Score of up to 16.79.

By Istiaque Ahmed, Afia Anjum Borsha, Ranat Das Prangon, Abu-fuad Ahmad, Thi Hong Tran
llmsragsafety
More like this →
arXiv AI
Oct 15

Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers

The paper evaluates TypeSafe AI’s Jev safety judge against GPT and GLM judges and local classifiers using a controlled adversarial test harness. Jev shows strong robustness on rule‑based tasks like input moderation, with minimal bypasses, but performs less well on judgment‑based tasks such as phishing detection, where it misses a higher proportion of phishing emails and is more prone to false positives. The study highlights Jev’s speed advantage and varying resilience depending on task type.

By Xiang Li, Aichun Huang, Mingming Zhang, Zheli Liu
llmssafety
More like this →