arXiv Computation and Language

AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Web Penetration Agents

AgentSnare is a trajectory‑adaptive deception system that expands a decoy environment while autonomous web‑penetration agents explore. It uses an LLM‑driven decoy constructor and a virtual kernel to generate and materialize decoy artifacts that persist across interactions, enabling long‑horizon exploration. In experiments on 15 CVE‑Bench web applications and four attacker models, AgentSnare delays 49.9% of tool calls, diverts 57.2% of post‑entry actions into decoys, and defuses 89.5% of completion reports, preventing any real target exploitation at pass@3 across 60 attacker‑CVE pairs.

arXiv Machine Learning
Sep 16

CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement

The paper introduces CoER, a framework that defends language‑model agents against adaptive indirect prompt injection (IPI) by employing attacker‑defender co‑evolution and refinement. CoER models IPI as a general‑sum Markov game, uses Co‑PPO to maintain historical opponent populations, and fine‑tunes defenders only on verified safe demonstrations. In experiments across seven domains, CoER cuts attack success from 38.5% to 0.2% while boosting task utility from 63.2% to 76.3%.

By Boyang Zhang, Qingxin Xiao, Lingwei Dang, Qingyao Wu
arXiv AI
Aug 18

SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system

arXiv:2608. 15012v1 Announce Type: cross Abstract: The rapid advancement of large language models (LLMs) has created a growing asymmetry in cybersecurity, where attack accelerates toward autonomous execution while defense remains predominantly human-intensive.

By Yuhan Meng, Shaofei Li, Jionghao Huang, Jiandong Jin, Puyi Wang, Hanlin Jiang, Anis Yusof, Peng Jiang, Zhenkai Liang, Yao Guo, Ding Li
arXiv AI
Jun 12

Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents

arXiv:2606. 13385v1 Announce Type: cross Abstract: Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments, where they operate over untrusted web content and execute actions with direct consequences.

By Zihao Wang, Yiming Li, Yutong Wu, Zheyu Liu, Kangjie Chen, Fok Kar Wai, Pin-Yu Chen, Vrizlynn L. L. Thing, Bo Li, Dacheng Tao, Tianwei Zhang
arXiv AI
Aug 28

How Do LLM Agents Actually Get the Flag? Trace-Level Provenance for Agentic Offensive Security Evaluation

The paper introduces CTF-ABACUS, a trace-based auditing framework that reconstructs each autonomous language-model agent’s run in Capture-the-Flag (CTF) challenges into evidence‑grounded solve profiles. By decomposing actions into penetration‑testing phases and techniques, it distinguishes genuine exploitation from shortcut methods such as memorized recall or guessing. Applying the framework to 1,435 CTF attempts by six models on 240 challenges shows that only 62‑87% of recovered flags are trace‑verified, highlighting that many successes rely on shallow trajectories rather than true exploitation.

By Kimberly Milner, Minghao Shao, Nanda Rani, Haoran Xi, Venkata Sai Charan Putrevu, Meet Udeshi, Sandeep K. Shukla, Prashanth Krishnamurthy, Farshad Khorrami, Muhammad Shafique, Ramesh Karri
arXiv Computation and Language
Oct 28

DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization

The paper introduces DYNASHIELD, a black-box moving target defense for large language models that dynamically customizes decoding hyperparameters and system prompts during inference. It first identifies decoding configurations that lower the probability of successful jailbreak attacks, then samples from a weighted pool of these configurations to add controlled variability to the model’s behavior. Experiments on seven open-source LLMs and four state-of-the-art jailbreak attacks show that DYNASHIELD significantly reduces attack success rates while preserving response quality and adding minimal inference overhead.

By Xiaoqun Liu, Weiming Qi, Qiben Yan
arXiv AI
Jun 8

EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attacks

arXiv:2505. 14289v2 Announce Type: replace Abstract: Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) are increasingly deployed yet vulnerable to Environmental Injection Attacks (EIAs).

By Yijie Lu, Manman Zhao, Tianjie Ju, Zihe Yan, Xinbei Ma, Yuan Guo, Daizong Ding, Gongshen Liu, Zhuosheng Zhang
arXiv AI
Sep 16

Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks

The paper proposes universal, tool‑based defenses for large language model agents that use external tools, addressing four types of adversarial attacks: direct and indirect prompt injection, memory poisoning, and backdoor attacks. Two main defenses are introduced: Attacker Tool Filtering, which uses anomaly detection to remove suspicious tools, and Normal Tool Recalling, which restores the agent’s original toolset before planning. The authors also add prompt‑based defenses such as Chain‑of‑Thought prompting and self‑reflection, and demonstrate that these methods dramatically lower attack success rates—often to 0%—across multiple open‑source and proprietary LLMs while maintaining or improving task performance.

By Xiaoyan Li, Yunli Wang