The paper introduces ERASE, a multimodal defense against indirect prompt injection that erases malicious instructions directly within the input media. By replacing the injected region with modality‑appropriate blank content and using on‑policy self‑distillation, ERASE trains a student model to learn from a frozen base model’s erased view, providing token‑level feedback during its own rollouts. Experiments on image, audio, and video tasks show near‑zero attack success while preserving task performance close to clean‑input levels, with significant improvements over prior methods such as ARGUS.
arXiv:2608.21500v1 Announce Type: cross Abstract: Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inje...
arXiv:2609. 09404v1 Announce Type: cross Abstract: Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services.
arXiv:2610.06401v2 Announce Type: replace-cross Abstract: Tool-using language-model agents are vulnerable to indirect prompt injection because they must act on untrusted external content. Existing tr...
arXiv:2609.36570v1 Announce Type: cross Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that...
The paper investigates how multimodal large language models are more susceptible to prompt injection when adversarial instructions are presented as text rather than as non-textual inputs like images. It proposes a training‑free defense that renders untrusted payloads into typographic images (or audio) before they reach the model, a method called Pictionary. Experiments on ten models and two benchmarks show that this approach significantly lowers attack success rates while maintaining normal functionality, and that fine‑tuning on image‑rendered instructions can further reduce the modality gap.
arXiv:2605. 08442v5 Announce Type: replace-cross Abstract: We discover that prompt-injection success and tool-execution success are separable safety properties: defenses that block injection do not necessarily block execution, and vice versa.
arXiv:2512. 02657v2 Announce Type: replace-cross Abstract: Real-world deployment of text-to-image diffusion models requires continual concept removal as new privacy, copyright, or safety obligations arise over time.
The paper introduces Checkerboard, a clean‑label backdoor attack that uses a closed‑form, data‑independent trigger design based on an input‑space Fisher‑separability objective and a ridge four‑neighbor local‑smoothness prior. This approach yields a pixel‑wise checkerboard trigger without requiring data access, surrogate model training, or iterative optimization, and it outperforms existing norm‑bounded clean‑label attacks across four benchmark datasets. On CIFAR‑10, poisoning 20 samples with a 10/255 perturbation achieves a 95.72% attack success rate, while on IN‑100 a 0.4% global poisoning rate yields over 83% ASR without harming clean accuracy, and the attack remains robust against state‑of‑the‑art defenses.
arXiv:2605. 08442v3 Announce Type: replace-cross Abstract: Persistent memory attacks against LLM agents achieve high attack success rates against open-source models.
arXiv:2611.07640v1 Announce Type: cross Abstract: Indirect prompt-injection (IPI) benchmarks predominantly test planted instructions that redirect a tool-using agent toward an attacker-chosen task. Y...
The paper introduces Semantic Overlays, a steering technique that adds non‑textual annotations to a language model’s input by applying learned adapters at specific prefill positions. These overlays create an out‑of‑band channel that encodes span identity and complex semantics, enabling the model to interpret marked text differently—such as rewriting code in a specified language or ignoring executable instructions. Experiments show that Semantic Overlays dramatically reduce prompt‑injection success rates while preserving model utility and readability of marked spans.
arXiv:2605.23954v2 Announce Type: replace-cross Abstract: Large Audio Language Models (LALMs) remain vulnerable to acoustic noise, which can obscure task-relevant evidence and produce unreliable resp...