The paper proposes universal, tool‑based defenses for large language model agents that use external tools, addressing four types of adversarial attacks: direct and indirect prompt injection, memory poisoning, and backdoor attacks. Two main defenses are introduced: Attacker Tool Filtering, which uses anomaly detection to remove suspicious tools, and Normal Tool Recalling, which restores the agent’s original toolset before planning. The authors also add prompt‑based defenses such as Chain‑of‑Thought prompting and self‑reflection, and demonstrate that these methods dramatically lower attack success rates—often to 0%—across multiple open‑source and proprietary LLMs while maintaining or improving task performance.
By Xiaoyan Li, Yunli Wang
The paper introduces ShadowMem, a defensive framework that protects large language model agents from long-horizon threats by maintaining a dedicated safety-focused memory. Inspired by the shadow stack concept, ShadowMem stores safety-critical context throughout an agent’s execution and uses this shadow memory to evaluate the risk of upcoming actions before they are carried out. Experiments show that ShadowMem outperforms existing defenses in detection accuracy, detects most attacks early, and adds minimal overhead to agent performance.
By Yuhui Wang, Tanqiu Jiang, Jiacheng Liang, Charles Fleming, Ting Wang
Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.
arXiv:2605.01970v4 Announce Type: replace-cross
Abstract: Memory systems enable otherwise stateless LLM agents to persist user information across sessions, but also introduce a new attack surface. Th...
By Debeshee Das, Julien Piet, Darya Kaviani, Luca Beurer-Kellner, Florian Tram\`er, David Wagner
arXiv:2606. 10749v1 Announce Type: cross Abstract: Large language model (LLM) agents are rapidly moving from conversational interfaces to software components that plan, invoke tools, maintain memory, and act on external environments.
By Yuchen Ling, Shengcheng Yu, Zhenyu Chen, Chunrong Fang
arXiv:2503. 15560v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly vulnerable to sophisticated multi-turn manipulation attacks, where adversaries strategically build context through seemingly benign conversational turns to circumvent safety measures and elicit harmful or unauthorized responses.
By Prashant Kulkarni, Assaf Namer
ActGov is a runtime enforcement framework that validates each action proposed by a large language model (LLM) agent before it interacts with external tools, ensuring that actions stay within task‑scoped authorization boundaries and comply with dynamically constructed policies. It builds policies from tool specifications, benign tasks, and failure traces, verifying updates via SMT‑based counterexample checking. In evaluations on AgentDojo and AgentDyn benchmarks, ActGov consistently reduces indirect prompt‑injection attack success while maintaining task utility, outperforming existing defenses.
By Kaiyuan Zhang, Yuke Peng, Ke Jiang, Yinqian Zhang
arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.
By Lipeng He, Yihan Wang, Jiawen Zhang, N. Asokan
arXiv:2607. 05189v1 Announce Type: cross Abstract: Persistent personal agents combine long-term memory with access to users' external environments, enabling personalized foreground assistance and proactive background execution.
By Yechao Zhang, Shiqian Zhao, Jiawen Zhang, Jie Zhang, Gelei Deng, Xiaogeng Liu, Chaowei Xiao, Tianwei Zhang
arXiv:2601. 09923v3 Announce Type: replace Abstract: AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior.
By Hanna Foerster, Tom Blanchard, Kristina Nikoli\'c, Ilia Shumailov, Cheng Zhang, Robert Mullins, Nicolas Papernot, Florian Tram\`er, Yiren Zhao
arXiv:2608. 15594v1 Announce Type: new Abstract: Multi-turn jailbreak attacks have emerged as a critical safety threat to LLMs, as harmful objectives are decomposed across a sequence of apparently benign turns to bypass guardrails.
By Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu, Ruihong Huang
arXiv:2508. 16481v3 Announce Type: replace Abstract: Ensuring the safe use of agentic systems requires a thorough understanding of the range of malicious behaviors these systems may exhibit.
By Jonathan N\"other, Adish Singla, Goran Radanovic