arXiv:2606. 27567v1 Announce Type: cross Abstract: Prompt injection is the top security risk for LLM-integrated applications, yet every defense proposed so far has been broken.
By Dewank Pant, Shruti Lohani, Avijit Kumar
The paper introduces a three‑layer security framework designed to protect retrieval‑augmented generation (RAG) chatbots from both direct and indirect prompt injection attacks. Layer 1 filters user input with rule‑based patterns and a semantic anomaly classifier; Layer 2 enforces a provenance‑based instruction hierarchy during context assembly; Layer 3 audits model output with a policy rule engine and semantic drift detector. Evaluations on GPT‑4o, Llama 3, and Mistral 7B demonstrate a reduction in attack success rate from 71.4 % to 11.3 %, outperforming existing single‑layer defenses while keeping false positives low and latency acceptable.
By Gulshan Saleem, Nisar Ahmed, Muhammad Imran Zaman, Ali Hassan, Umar Mujahid
arXiv:2510. 10271v2 Announce Type: replace-cross Abstract: Unlike regular tokens derived from existing text corpora, special tokens are artificially created to annotate structured conversations during the fine-tuning process of Large Language Models (LLMs).
By Wentian Zhu, Zhen Xiang, Wei Niu, Le Guan
arXiv:2608. 07808v1 Announce Type: cross Abstract: Four years after prompt injection was first identified in 2022, attacks are still predominantly documented as verbatim strings rather than structured exploits, despite advancing agent capabilities and threat actors embedding injections to subvert AI-assisted security analysis.
By Jeremy McHugh
The paper introduces Learnable Trust‑Boundary Delimiters (LTBD), a lightweight defense that embeds learnable delimiters into LLM inputs to explicitly mark trusted user instructions versus untrusted external data. LTBD keeps the model’s parameters unchanged, achieving superior performance over inference‑time defenses and matching training‑based methods while adding negligible inference overhead. Experiments show near‑zero attack success rates on AlpacaFarm and TaskTracker, and the method remains robust even against adaptive attackers who know the defense.
By Luman Zhao, Minghui Xu, Yue Zhang, Yijun Yang
arXiv:2503. 24191v4 Announce Type: replace-cross Abstract: Content Warning: This paper may contain unsafe or harmful content generated by LLMs that may be offensive to readers.
By Shuoming Zhang, Jiacheng Zhao, Hanyuan Dong, Ruiyuan Xu, Zhicheng Li, Yangyu Zhang, Shuaijiang Li, Yuan Wen, Chunwei Xia, Zheng Wang, Xiaobing Feng, Huimin Cui