arXiv AI By Xinting Liao, Behnoosh Zamanlooy, Masoumeh Shafieinejad, David B. Emerson, Ruinan Jin, Deval Pandya, Xiaoxiao Li

CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization

Read the original on arXiv AI →

arXiv:2607. 18622v1 Announce Type: cross Abstract: Textual Collaborative Prompt Optimization (TCPO) extends Textgrad (Yuksekgonul et al.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 26

Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors

The paper introduces Semantic Overlays, a steering technique that adds non‑textual annotations to a language model’s input by applying learned adapters at specific prefill positions. These overlays create an out‑of‑band channel that encodes span identity and complex semantics, enabling the model to interpret marked text differently—such as rewriting code in a specified language or ignoring executable instructions. Experiments show that Semantic Overlays dramatically reduce prompt‑injection success rates while preserving model utility and readability of marked spans.

By Joshua Penman
arXiv AI
Sep 16

Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks

The paper proposes universal, tool‑based defenses for large language model agents that use external tools, addressing four types of adversarial attacks: direct and indirect prompt injection, memory poisoning, and backdoor attacks. Two main defenses are introduced: Attacker Tool Filtering, which uses anomaly detection to remove suspicious tools, and Normal Tool Recalling, which restores the agent’s original toolset before planning. The authors also add prompt‑based defenses such as Chain‑of‑Thought prompting and self‑reflection, and demonstrate that these methods dramatically lower attack success rates—often to 0%—across multiple open‑source and proprietary LLMs while maintaining or improving task performance.

By Xiaoyan Li, Yunli Wang