The paper introduces a three‑layer security framework designed to protect retrieval‑augmented generation (RAG) chatbots from both direct and indirect prompt injection attacks. Layer 1 filters user input with rule‑based patterns and a semantic anomaly classifier; Layer 2 enforces a provenance‑based instruction hierarchy during context assembly; Layer 3 audits model output with a policy rule engine and semantic drift detector. Evaluations on GPT‑4o, Llama 3, and Mistral 7B demonstrate a reduction in attack success rate from 71.4 % to 11.3 %, outperforming existing single‑layer defenses while keeping false positives low and latency acceptable.
By Gulshan Saleem, Nisar Ahmed, Muhammad Imran Zaman, Ali Hassan, Umar Mujahid
arXiv:2610.28364v1 Announce Type: cross
Abstract: Local Retrieval-Augmented Generation (RAG) systems are vulnerable to indirect prompt injection and retrieval poisoning, where malicious instructions...
By Abhishek Rai
arXiv:2606. 15609v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly rely on long-term memory to support complex task execution, user personalization, and domain adaptation.
By Zixin Rao, Wentian Zhu, Chan Aristella Lu, Zhaorun Chen, Wei Niu, Le Guan, Bo Li, Zhen Xiang
arXiv:2606. 20470v1 Announce Type: cross Abstract: Agentic AI systems increasingly rely on language-model components to interpret instructions, process external data, invoke tools, and coordinate with other agents.
By Reza Soosahabi, Vivek Namsani
arXiv:2605. 01143v2 Announce Type: replace Abstract: Large Language Model (LLM)-powered agents demonstrate strong capabilities in autonomous task execution, tool use, and multi-step reasoning.
By Sheldon Yu, Yingcheng Sun, Hanqing Guo, Qianqian Tong
arXiv:2608. 05430v1 Announce Type: cross Abstract: The remarkable instruction-following ability of modern LLMs has enabled their practical use as the minds of agents that can autonomously complete increasingly complex tasks.
By Buzhao Liu, Xinhang Ma, Yevgeniy Vorobeychik
Reflex-Guard is a lightweight, locally‑run guardrail for large language models that uses jailbreak‑aware preprocessing, compact sentence‑transformer embeddings, and seven fast binary classifiers to filter unsafe prompts. It achieves 95.9 % recall on harmful prompts with an end‑to‑end latency of 37.6 ms, outperforming existing solutions such as Llama Guard 2 (255 ms) and SafeDecoding (723 ms). The system can detect all GCG suffix attacks and Base64‑encoded prompts at the default threshold, while DrAttack structured prompts require a lower threshold of 0.03 for optimal detection, and it attains a Reflex Efficiency Score of up to 16.79.
By Istiaque Ahmed, Afia Anjum Borsha, Ranat Das Prangon, Abu-fuad Ahmad, Thi Hong Tran
arXiv:2510. 01529v3 Announce Type: replace Abstract: Ball et al.
By Jaiden Fairoze, Sanjam Garg, Keewoo Lee, Mingyuan Wang
Reflex-Guard is a lightweight, locally running guardrail for large language models that uses jailbreak-aware preprocessing, compact sentence‑transformer embeddings, and seven fast binary classifiers to filter unsafe prompts. It achieves 95.9% recall on harmful prompts with an end‑to‑end latency of 37.6 ms, far faster than existing solutions such as Llama Guard 2 (255 ms) and SafeDecoding (723 ms). The system can detect all GCG suffix attacks and Base64‑encoded prompts at the default threshold, and it attains a Reflex Efficiency Score up to 16.79, outperforming its competitors.
RAG-CT is a defense mechanism designed to protect Retrieval-Augmented Generation (RAG) systems from leaking personally identifiable information (PII). It works by detecting malicious queries through analysis of entropy and margin distributions, applying a score-based detection method. Experiments across four attack strategies and four baseline defenses on two datasets show that RAG-CT significantly reduces PII leakage while outperforming existing defenses, all without altering the underlying LLM or retriever.
By Xingyu Lyu, Jiayimei Wang, Jianfeng He, Ning Wang, Yidan Hu, Yimin Chen
arXiv:2607. 00481v1 Announce Type: cross Abstract: Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs).
By Junlong Liu, Haobo Wang, Weiqi Luo, Xiaojun Jia
The paper examines how fine‑tuned defenses against prompt injection in large language models can be audited before deployment. By leveraging intermediate checkpoints from the fine‑tuning process, the authors create strong optimization‑based attacks (e.g., GCG and BEAST) that dramatically increase attack success rates—from near zero to up to 96%—against state‑of‑the‑art defenses. This demonstrates that current empirical evaluations often overestimate robustness and that pre‑deployment stress‑testing can uncover real vulnerabilities.
By Xiaoxue Yang, Bozhidar Stevanoski, Matthieu Meeus, Yves-Alexandre de Montjoye