arXiv AI
Detecting and Mitigating Query-Based System Prompt Leakage Attacks
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Aug 27
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
The paper introduces a three‑layer security framework designed to protect retrieval‑augmented generation (RAG) chatbots from both direct and indirect prompt injection attacks. Layer 1 filters user input with rule‑based patterns and a semantic anomaly classifier; Layer 2 enforces a provenance‑based instruction hierarchy during context assembly; Layer 3 audits model output with a policy rule engine and semantic drift detector. Evaluations on GPT‑4o, Llama 3, and Mistral 7B demonstrate a reduction in attack success rate from 71.4 % to 11.3 %, outperforming existing single‑layer defenses while keeping false positives low and latency acceptable.
arXiv AI
Oct 21
Context-Aware Prompt Injection Detection via Small Language Models in RAG Pipelines: nokast-secureRAG
arXiv:2610.28364v1 Announce Type: cross Abstract: Local Retrieval-Augmented Generation (RAG) systems are vulnerable to indirect prompt injection and retrieval poisoning, where malicious instructions...
arXiv AI
Jun 16
FragFuse: Bypassing Access Control of Large Language Model Agents via Memory-Based Query Fragmentation and Fusion
arXiv:2606. 15609v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly rely on long-term memory to support complex task execution, user personalization, and domain adaptation.
arXiv AI
Jun 19
Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systems
arXiv:2606. 20470v1 Announce Type: cross Abstract: Agentic AI systems increasingly rely on language-model components to interpret instructions, process external data, invoke tools, and coordinate with other agents.
arXiv AI
Jul 14
A Low-Latency Fraud Detection Layer for Detecting Adversarial Interaction Patterns in LLM-Powered Agents
arXiv:2605. 01143v2 Announce Type: replace Abstract: Large Language Model (LLM)-powered agents demonstrate strong capabilities in autonomous task execution, tool use, and multi-step reasoning.
arXiv Machine Learning
Aug 7
Robust Context-Aware Detection of Malicious Instructions in Text
arXiv:2608. 05430v1 Announce Type: cross Abstract: The remarkable instruction-following ability of modern LLMs has enabled their practical use as the minds of agents that can autonomously complete increasingly complex tasks.