ActGov is a runtime enforcement framework that validates each action proposed by a large language model (LLM) agent before it interacts with external tools, ensuring that actions stay within task‑scoped authorization boundaries and comply with dynamically constructed policies. It builds policies from tool specifications, benign tasks, and failure traces, verifying updates via SMT‑based counterexample checking. In evaluations on AgentDojo and AgentDyn benchmarks, ActGov consistently reduces indirect prompt‑injection attack success while maintaining task utility, outperforming existing defenses.
By Kaiyuan Zhang, Yuke Peng, Ke Jiang, Yinqian Zhang
Large language model (LLM) agents increasingly execute long-horizon workflows through external tools, allowing untrusted outputs to influence subsequent actions and exceed user authorization. Existing...
The paper proposes universal, tool‑based defenses for large language model agents that use external tools, addressing four types of adversarial attacks: direct and indirect prompt injection, memory poisoning, and backdoor attacks. Two main defenses are introduced: Attacker Tool Filtering, which uses anomaly detection to remove suspicious tools, and Normal Tool Recalling, which restores the agent’s original toolset before planning. The authors also add prompt‑based defenses such as Chain‑of‑Thought prompting and self‑reflection, and demonstrate that these methods dramatically lower attack success rates—often to 0%—across multiple open‑source and proprietary LLMs while maintaining or improving task performance.
By Xiaoyan Li, Yunli Wang
arXiv:2609.39450v1 Announce Type: cross
Abstract: LLM-based agents extend their capabilities through third-party skills that provide task-specific instructions, scripts, and tool-use procedures. Howe...
By Jihun Han, Yejin Jang, Byung Il Kwak, Mee Lan Han
arXiv:2609.36879v1 Announce Type: cross
Abstract: As LLM-based agents perform increasingly complex tasks, Agent Skills have emerged as a flexible mechanism for extending their capabilities. An Agent...
By Haoran Ou, Gelei Deng, Xuanye Zhang, Wenbo Guo, Tianwei Zhang, Kwok-Yan Lam
arXiv:2509. 25624v3 Announce Type: replace-cross Abstract: As LLMs advance into autonomous agents with tool-use capabilities, they introduce security challenges that extend beyond traditional content-based LLM safety concerns.
By Jing-Jing Li, Jianfeng He, Chao Shang, Devang Kulshreshtha, Xun Xian, Yi Zhang, Hang Su, Sandesh Swamy, Yanjun Qi
AgentSentry is an inference‑time defense for tool‑augmented large language model agents that mitigates indirect prompt injection (IPI) attacks. It uses temporal causal diagnostics by performing counterfactual re‑executions at tool‑return boundaries to detect mediator‑sensitive proposals, purifies implicated content, and resumes execution from a sanitized state. Evaluations on AgentDojo show zero successful attacks (ASR = 0%) and an average Utility Under Attack of 76.10%, outperforming baselines while maintaining low false‑positive rates.
By Tian Zhang, Yiwei Xu, Juan Wang, Keyan Guo, Xiaoyang Xu, Bowen Xiao, Quanlong Guan, Jinlin Fan, Jiawei Liu, Zhiquan Liu, Hongxin Hu
INFUSE is a new benchmark that tests indirect prompt injection (IPI) in large‑language‑model agents performing long‑horizon workflows. It contains 40 realistic tasks and 56 attack scenarios across five threat categories, with attacks injected through realistic interaction surfaces rather than just initial prompts. The study evaluated 19 LLMs, generating over 41,000 adversarial trajectories and found mean attack success rates of 19.6% (LLM evaluation) and 15.4% (heuristic verification), while prompt‑level hardening reduced success from 58.2% to 14.3%.
By Thuy Dung Nguyen, Jingyi Guo, Rana Seda Dubauskas, Sreepriya Damuluru, Taylor T. Johnson, Kevin Leach
The paper introduces a unified framework for monitoring misuse of large language model agents in multi‑actor settings, focusing on two attack types: decomposition attacks and prompt injection attacks. It proposes tracking the agent’s externalised actions and determining whether the first detection of harm falls within a defined harm window. Using this formalism, the authors build a benchmark of about 6,200 conversation transcripts and evaluate 17 monitor configurations, finding that action‑framed monitors perform well on both threats while content‑framed monitors struggle with injection attacks, and highlighting limitations of traditional position‑blind metrics.
By Aniruddh Pramod, James Oldfield, Adel Bibi
arXiv:2606. 15057v1 Announce Type: cross Abstract: Indirect prompt injection (IPI) is a major security threat to LLM-powered agents.
By Xinhang Ma, Taoran Li, Chaowei Xiao, Zhiyuan Yu, Ning Zhang, Yevgeniy Vorobeychik
arXiv:2607. 25297v1 Announce Type: cross Abstract: The rapid development of large language model (LLM) agents has enabled their broad adoption across diverse real-world tasks.
By Ping He, Yuexiang Xie, Yaliang Li, Shouling Ji
The paper examines how fine‑tuned defenses against prompt injection in large language models can be audited before deployment. By leveraging intermediate checkpoints from the fine‑tuning process, the authors create strong optimization‑based attacks (e.g., GCG and BEAST) that dramatically increase attack success rates—from near zero to up to 96%—against state‑of‑the‑art defenses. This demonstrates that current empirical evaluations often overestimate robustness and that pre‑deployment stress‑testing can uncover real vulnerabilities.
By Xiaoxue Yang, Bozhidar Stevanoski, Matthieu Meeus, Yves-Alexandre de Montjoye