arXiv AI
Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 16
Nameless Tokenization: A Lossless Tokenizer-Level Defense Against Control-Token Forgery in Open-Weight LLMs
The paper examines how open‑weight language models expose the control tokens used in chat templates, allowing attackers to forge turn boundaries that the model treats as legitimate. An audit of 256 deployed tokenizers shows all are vulnerable, and the commonly recommended flag fails to protect 56.6% of cases. The authors introduce nameless tokenization, which removes surface strings for control identifiers while preserving their internal representation, achieving identical token streams on clean data and significantly improving accuracy on delimiter‑bearing text.
arXiv AI
Aug 26
Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors
The paper introduces Semantic Overlays, a steering technique that adds non‑textual annotations to a language model’s input by applying learned adapters at specific prefill positions. These overlays create an out‑of‑band channel that encodes span identity and complex semantics, enabling the model to interpret marked text differently—such as rewriting code in a specified language or ignoring executable instructions. Experiments show that Semantic Overlays dramatically reduce prompt‑injection success rates while preserving model utility and readability of marked spans.
arXiv AI
Jun 17
Structural Role Injection in Handlebars-Templated LLM Prompts: Triple-Brace Interpolation, Delimiter Family, and the Limits of HTML Auto-Escaping
arXiv:2606. 18120v1 Announce Type: cross Abstract: Large language model applications build prompts from templates, and Handlebars is a widely used templating engine and the default prompt-template format in Microsoft Semantic Kernel.
arXiv AI
Aug 25
SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation
arXiv:2608.21500v1 Announce Type: cross Abstract: Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inje...
arXiv AI
4d ago
CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering
arXiv:2609.36570v1 Announce Type: cross Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that...
arXiv Machine Learning
Jun 5
SlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attacks
arXiv:2606. 05609v1 Announce Type: cross Abstract: As large language models (LLMs) are widely deployed, identifying their vulnerability through jailbreak attacks becomes increasingly critical.