arXiv:2606.17467v3 Announce Type: replace-cross Abstract: Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and int...
TriShieldRAG introduces a three‑layered defense for Retrieval‑Augmented Generation: an Ingest Guard that screens documents, a Retrieval Scorer that re‑ranks based on trust, and a Cross‑LLM Consensus that validates evidence across three models. Against the original PoisonedRAG attack on the 2.68M‑passage Natural Questions corpus, the framework reduces attack success from about 79% to 1%. However, adaptive attacks that only alter document formatting can bypass the Ingest Guard and still achieve high success rates, revealing limits of layered defenses that rely on the same retrieved evidence.
arXiv:2608. 02678v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) systems are vulnerable to corpus poisoning: an attacker who inserts a crafted document into the retrieval corpus can steer the underlying large language model (LLM) toward an attacker-chosen wrong answer.
arXiv:2509. 20324v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) is an emerging approach in natural language processing that combines large language models (LLMs) with external document retrieval to produce more accurate and grounded responses.
arXiv:2604. 08304v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) extends large language models (LLMs) with external knowledge, but this access path also introduces security risks that existing work often conflates with inherent LLM flaws.
arXiv:2608. 09001v1 Announce Type: cross Abstract: Black-box privacy scores for retrieval-augmented generation (RAG) are difficult to interpret unless the audited defense's active pipeline hook is known.
The paper demonstrates that fine‑tuning large language models on a single author’s works can trigger the models to reproduce large verbatim excerpts from copyrighted books, even when prompted only with semantic descriptions. Experiments on GPT‑4o, Gemini‑2.5‑Pro, and DeepSeek‑V3.1 show up to 85‑90% recall of held‑out books, with spans exceeding 460 words, and this effect generalizes across authors and model providers. The findings suggest that fine‑tuning reactivates latent memorization from pre‑training, revealing a widespread vulnerability in industry models.
QuanText is a training‑free, large‑language‑model‑agnostic mechanism for releasing textual datasets that protects dataset‑level secrets such as the proportion of records with a particular diagnosis or gender. It perturbs both the secret distribution and correlated attribute distributions by selecting candidate release distributions close to the private empirical distribution and rewriting each text sample to match the chosen distribution using attribute‑related snippets. The method is inspired by the Statistic Maximal Leakage framework and, under idealized conditions, satisfies an SML guarantee, while empirical evaluations show a superior privacy‑utility trade‑off compared to existing data generation baselines.
RAG-CT is a defense mechanism designed to protect Retrieval-Augmented Generation (RAG) systems from leaking personally identifiable information (PII). It works by detecting malicious queries through analysis of entropy and margin distributions, applying a score-based detection method. Experiments across four attack strategies and four baseline defenses on two datasets show that RAG-CT significantly reduces PII leakage while outperforming existing defenses, all without altering the underlying LLM or retriever.
The paper surveys attacks and defenses in Retrieval-Augmented Generation (RAG), a technique that improves large language models by grounding outputs in external knowledge. It identifies new robustness and security risks such as corpus poisoning, backdoor attacks, privacy leakage, and fairness violations, and notes that existing surveys inadequately cover attacker objectives, threat models, and stage-specific defenses. The survey offers a unified, pipeline-aware overview, formalizing threat models across the corpus, retriever, and generator, categorizing attacks by accuracy, privacy, and fairness, and reviewing defenses for retrieval, rerank, generation, and traceback stages, while also summarizing robustness benchmarks and explainability methods.
RAG-Safety-Bench is a benchmark designed to evaluate how retrieval-augmented generation (RAG) affects the safety of large language models (LLMs). It isolates safety impacts by testing four conditions: non-RAG, RAG with an oracle document, RAG with related but non-answer documents, and RAG with random safe documents. Results on five open-source LLMs reveal an inverse relationship between benign and unsafe capabilities, show that baseline safety guardrails do not guarantee safety in RAG, and confirm that even benign documents can trigger unsafe generation.
arXiv:2502.10673v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) has become an effective method for enhancing large language models (LLMs) with up-to-date knowledge. How...