arXiv AI
Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 2
Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing
arXiv:2606. 00686v1 Announce Type: new Abstract: The prevailing paradigm in large language model (LLM) alignment operates via erasure, filtering unsafe data or training models to strictly refuse harmful prompts.
arXiv AI
Aug 6
Breadcrumbing Search Agents
arXiv:2608. 04565v1 Announce Type: cross Abstract: LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk: web content retrieved during execution is untrusted, exposing agents to prompt injection and goal hijacking.
arXiv AI
Aug 18
The Synthetic Web: Adversarially-Curated Mini-Internets for Diagnosing Epistemic Weaknesses of Language Agents
arXiv:2603. 00801v2 Announce Type: replace Abstract: Language agents increasingly act as web-enabled systems that search, browse, and synthesize information from diverse sources.
arXiv Computation and Language
1d ago
RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
RAG-Safety-Bench is a benchmark designed to evaluate how retrieval-augmented generation (RAG) affects the safety of large language models (LLMs). It isolates safety impacts by testing four conditions: non-RAG, RAG with an oracle document, RAG with related but non-answer documents, and RAG with random safe documents. Results on five open-source LLMs reveal an inverse relationship between benign and unsafe capabilities, show that baseline safety guardrails do not guarantee safety in RAG, and confirm that even benign documents can trigger unsafe generation.
Hugging Face Trending Papers
2d ago
RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-au...
arXiv Machine Learning
Aug 27
Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
The paper surveys attacks and defenses in Retrieval-Augmented Generation (RAG), a technique that improves large language models by grounding outputs in external knowledge. It identifies new robustness and security risks such as corpus poisoning, backdoor attacks, privacy leakage, and fairness violations, and notes that existing surveys inadequately cover attacker objectives, threat models, and stage-specific defenses. The survey offers a unified, pipeline-aware overview, formalizing threat models across the corpus, retriever, and generator, categorizing attacks by accuracy, privacy, and fairness, and reviewing defenses for retrieval, rerank, generation, and traceback stages, while also summarizing robustness benchmarks and explainability methods.