Hugging Face Trending Papers
Detecting and Mitigating Query-Based System Prompt Leakage Attacks
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Oct 28
Detecting and Mitigating Query-Based System Prompt Leakage Attacks
The paper introduces AdvGuard, a model‑agnostic framework designed to detect and mitigate query‑based system prompt leakage attacks against large language models. AdvGuard constructs a bounded semantic interaction graph to capture cross‑query relationships, applies statistical filtering, and uses graph convolutional networks for classification, enabling it to identify coordinated extraction attempts across multiple turns. In evaluations, AdvGuard achieves significant improvements in recall and F1 scores over baselines, with negligible false positives, and effectively limits prompt leakage in adaptive attack scenarios.
arXiv Computation and Language
Aug 27
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
The paper introduces a three‑layer security framework designed to protect retrieval‑augmented generation (RAG) chatbots from both direct and indirect prompt injection attacks. Layer 1 filters user input with rule‑based patterns and a semantic anomaly classifier; Layer 2 enforces a provenance‑based instruction hierarchy during context assembly; Layer 3 audits model output with a policy rule engine and semantic drift detector. Evaluations on GPT‑4o, Llama 3, and Mistral 7B demonstrate a reduction in attack success rate from 71.4 % to 11.3 %, outperforming existing single‑layer defenses while keeping false positives low and latency acceptable.
arXiv Machine Learning
Jun 4
Bypassing Prompt Guards in Production with Controlled-Release Prompting
arXiv:2510. 01529v3 Announce Type: replace Abstract: Ball et al.
arXiv AI
Jun 19
Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systems
arXiv:2606. 20470v1 Announce Type: cross Abstract: Agentic AI systems increasingly rely on language-model components to interpret instructions, process external data, invoke tools, and coordinate with other agents.
arXiv AI
Jul 14
A Low-Latency Fraud Detection Layer for Detecting Adversarial Interaction Patterns in LLM-Powered Agents
arXiv:2605. 01143v2 Announce Type: replace Abstract: Large Language Model (LLM)-powered agents demonstrate strong capabilities in autonomous task execution, tool use, and multi-step reasoning.
Hugging Face Trending Papers
Aug 18
Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings
Reflex-Guard is a lightweight, locally running guardrail for large language models that uses jailbreak-aware preprocessing, compact sentence‑transformer embeddings, and seven fast binary classifiers to filter unsafe prompts. It achieves 95.9% recall on harmful prompts with an end‑to‑end latency of 37.6 ms, far faster than existing solutions such as Llama Guard 2 (255 ms) and SafeDecoding (723 ms). The system can detect all GCG suffix attacks and Base64‑encoded prompts at the default threshold, and it attains a Reflex Efficiency Score up to 16.79, outperforming its competitors.