ENDOPROMPT is a white‑box technique that learns prefixes to degrade the utility of language model responses without producing harmful content. It uses clean victim continuations as pseudo‑references, performing local search and preference fitting on unlabeled instructions to generate utility‑degrading prefixes. In experiments on four instruction‑tuned models and seven benign benchmarks, ENDOPROMPT reduces utility by an average of 26.8 percentage points, with 27 of 28 benchmark cells showing negative change.
arXiv:2602. 05746v2 Announce Type: replace-cross Abstract: Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts.
By Xin Chen, Jie Zhang, Florian Tram\`er
arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.
By Lipeng He, Yihan Wang, Jiawen Zhang, N. Asokan
The paper introduces a new benchmark for indirect prompt injection (IPI) that focuses on attacks which tamper with decisions already in progress rather than redirecting the task. In 90 validated cases across three tool‑use environments, attackers target up to six aspects of an agent’s decision‑making—objective, decision rules, available actions, expected consequences, observation reading, or state beliefs—while keeping the original task intact. The study finds that many defenses reduce objective‑targeting attacks but leave observation and state attacks largely effective, highlighting gaps in current mitigation strategies.
By Rongting Zhang, Martin Bertran, Sandeep Avula, Chia-Jung Lee, Vanessa Murdock
Guardrail models, which screen malicious prompts in LLM services, often use lightweight Transformers with short context windows and bucketed positional encodings. The study identifies a new failure mode called Overflip, where repeating a prompt causes the guardrail’s prediction to flip from malicious to benign as the sequence length increases. Experiments on nine popular guardrails show that 5 models exhibit MAL→BEN flips on 100 prompts, with flip rates ranging from 8% to 92% and first flips occurring between 2.6k and 9.4k tokens, highlighting a gradual attention dispersion distinct from traditional attention‑dilution attacks.
By Xu He, Chih-Hsuan Lin, Hung-Mao Chen, Junjie Xiong, Yan Zhai, Kun Sun
Large Language Models (LLMs) treat all context tokens uniformly, which creates prompt injection vulnerabilities and unreliable instruction execution. The paper introduces LLMON (LLM Object Notation), a post‑training framework that separates executable instructions from passive data and requires explicit identifier‑based execution. Using a 4.6 B‑token structured corpus and post‑training on Granite‑4.1 models, LLMON reduces indirect prompt‑injection success rates from 53–100 % to 0.02–5.6 % while maintaining general capabilities, and provides attention‑based interpretability metrics to diagnose failure modes.
By Basel Shbita, Michael Hind, Chad DeLuca, Nathan Fulton, Shubhi Asthana, Farhan Ahmed, Anisa Halimi, Yada Zhu, Mauro Martino, David Cox, Dan Gutfreund