arXiv:2606. 30783v1 Announce Type: cross Abstract: We identify a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses resist injected instructions largely by suppressing untrusted text, which corrupts tasks that must preserve it, such as translation and document editing.
arXiv:2611.07640v1 Announce Type: cross Abstract: Indirect prompt-injection (IPI) benchmarks predominantly test planted instructions that redirect a tool-using agent toward an attacker-chosen task. Y...
arXiv:2608.21500v1 Announce Type: cross Abstract: Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inje...
The paper introduces APEX, an active defense for large language model agents that protects against indirect prompt injection by enforcing safety at execution boundaries. APEX uses an evidence‑gated prevention contract and deception‑based exposure to ensure that only authorized effects, endorsed by the task, are executed. Evaluation shows APEX achieves near‑zero attack success across multiple benchmarks and capability‑unit types, outperforming 13 baseline defenses.
arXiv:2610. 37657v1 Announce Type: cross Abstract: We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates.
arXiv:2609.36570v1 Announce Type: cross Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that...
SecPO introduces a principled adversarial training framework to defend large language models against prompt injection attacks. By approximating an ideal injection‑invariant reference model and employing a new SecPO loss, the authors improve security while preserving utility. Combined with an efficient adversarial training pipeline that synthesizes strong injected inputs, the resulting SecPO + AT model significantly reduces attack success rates compared to existing defenses and maintains performance across multiple benchmarks.
The paper examines how fine‑tuned defenses against prompt injection in large language models can be audited before deployment. By leveraging intermediate checkpoints from the fine‑tuning process, the authors create strong optimization‑based attacks (e.g., GCG and BEAST) that dramatically increase attack success rates—from near zero to up to 96%—against state‑of‑the‑art defenses. This demonstrates that current empirical evaluations often overestimate robustness and that pre‑deployment stress‑testing can uncover real vulnerabilities.
SentryBench is a cost‑sensitive benchmark designed to evaluate large language model agents that use tools in public‑safety contexts. It measures agents on a frontier that jointly considers attack success, over‑refusal (missing real emergencies), and over‑reaction (unwarranted actions) under adversarial tool use, using scenarios grounded in real emergency‑call transcripts and disaster messages. The study shows that naive caution leads to high harm, while a targeted trust‑gate wrapper reduces harm across models and defenses.
arXiv:2609.22818v1 Announce Type: cross Abstract: Memory-poisoning defenses for LLM agents are typically evaluated by their ability to prevent attacks. However, the traffic they process is rarely adv...
arXiv:2606. 18530v1 Announce Type: cross Abstract: Domain-camouflaged injection attacks embed malicious instructions in retrieved content using domain-appropriate vocabulary, evading standard detectors that rely on syntactic injection markers.
The paper investigates typed decision models used as guardrails in agent systems, evaluating seven open‑weight models on tasks such as prompt‑injection, jailbreak, and toxic‑content screening. Accuracy for allow‑or‑block decisions ranges from 36% to 72% above chance, with models either over‑allowing or over‑blocking. The authors demonstrate that simple textual manipulations—like adding misleading names or server log lines—can dramatically increase fail‑open rates, and that all tested defenses can be bypassed, suggesting these models are unsuitable as the sole decision component.
arXiv:2609.15017v1 Announce Type: cross Abstract: Prompt-injection detectors are typically evaluated using aggregate F1 on in-distribution test data, which offers limited insight into behavior under...