arXiv AI
SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 26
Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents
arXiv:2606. 26479v1 Announce Type: cross Abstract: Recent work (2024 to 2026) has converged on a strategy for defending tool-using LLM agents against indirect prompt injection: rather than training the model to refuse malicious instructions, enforce security outside the model with a deterministic policy that mediates the agent's actions.
arXiv AI
Jun 16
AutoDojo: Adaptive Attacks Expose Superficial Defenses and User-Underspecification Limits in LLM Agents
arXiv:2606. 15057v1 Announce Type: cross Abstract: Indirect prompt injection (IPI) is a major security threat to LLM-powered agents.
arXiv AI
4d ago
Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors
The paper introduces Semantic Overlays, a steering technique that adds non‑textual annotations to a language model’s input by applying learned adapters at specific prefill positions. These overlays create an out‑of‑band channel that encodes span identity and complex semantics, enabling the model to interpret marked text differently—such as rewriting code in a specified language or ignoring executable instructions. Experiments show that Semantic Overlays dramatically reduce prompt‑injection success rates while preserving model utility and readability of marked spans.
arXiv Machine Learning
Jul 24
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.
arXiv AI
Jun 16
Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.
Hugging Face Trending Papers
Jun 13
Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.