ENDOPROMPT is a white‑box attack that learns prefixes to degrade the utility of language model responses without producing harmful content. It uses clean victim continuations as pseudo‑references, performing local search and preference fitting on unlabeled instructions to distill a generator that outputs a single prefix per request. In experiments across four instruction‑tuned models and seven benign benchmarks, ENDOPROMPT reduces utility by an average of 26.8 percentage points, with 27 of 28 test cells showing negative change.
By Qingyu Wu, Zeyu Feng, Yongda Yu, Yuzhe Luo, Hua Cheng
Large Language Models (LLMs) treat all context tokens uniformly, which creates prompt injection vulnerabilities and unreliable instruction execution. The paper introduces LLMON (LLM Object Notation), a post‑training framework that separates executable instructions from passive data and requires explicit identifier‑based execution. Using a 4.6 B‑token structured corpus and post‑training on Granite‑4.1 models, LLMON reduces indirect prompt‑injection success rates from 53–100 % to 0.02–5.6 % while maintaining general capabilities, and provides attention‑based interpretability metrics to diagnose failure modes.
By Basel Shbita, Michael Hind, Chad DeLuca, Nathan Fulton, Shubhi Asthana, Farhan Ahmed, Anisa Halimi, Yada Zhu, Mauro Martino, David Cox, Dan Gutfreund
arXiv:2602. 05746v2 Announce Type: replace-cross Abstract: Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts.
By Xin Chen, Jie Zhang, Florian Tram\`er
Guardrail models, which screen malicious prompts in LLM services, often use lightweight Transformers with short context windows and bucketed positional encodings. The study identifies a new failure mode called Overflip, where repeating a prompt causes the guardrail’s prediction to flip from malicious to benign as the sequence length increases. Experiments on nine popular guardrails show that 5 models exhibit MAL→BEN flips on 100 prompts, with flip rates ranging from 8% to 92% and first flips occurring between 2.6k and 9.4k tokens, highlighting a gradual attention dispersion distinct from traditional attention‑dilution attacks.
By Xu He, Chih-Hsuan Lin, Hung-Mao Chen, Junjie Xiong, Yan Zhai, Kun Sun
arXiv:2610.06401v2 Announce Type: replace-cross
Abstract: Tool-using language-model agents are vulnerable to indirect prompt injection because they must act on untrusted external content. Existing tr...
By Mohamed Dhouib, Clement Elliker, Alexi Canesse, Ma\"el Jenny, Lucas-Andrei Thil, Mahammed El Sharkawy, Sonia Vanier, Elie Bursztein
arXiv:2609.36570v1 Announce Type: cross
Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that...
By Mark Russinovich