The study compares deterministic static application security testing (SAST) with AI-powered coding agents and security harnesses on the VulnBench 2.0 benchmark, which contains 108 curated vulnerabilities across 20 open‑source web applications. Nine productized configurations—including Snyk Code, Claude Code, Codex Security, and DeepSec—were evaluated under a blind protocol, with Snyk Code achieving 82.4% macro recall and Claude Code 79.3%. Together, the AI configurations identified 101 of the 108 vulnerabilities, surpassing any single tool, and also uncovered seven additional real vulnerabilities not present in the reference set, highlighting the complementary strengths of AI harnesses and deterministic SAST.
By Liran Tal, Chris Liu, Charlotte Fereday, Sebastian Roth, Brian Clark
arXiv:2608. 11469v1 Announce Type: cross Abstract: AI agents are rapidly improving in cybersecurity capabilities when the source code is available for analysis, yet much of the software most consequential to cybersecurity, including malware, firmware, and proprietary applications, is available only as binaries.
By Jeremy Spence, Nicholas Assaderaghi, Jinhao Zhu, Nikil Ravi, Raluca Ada Popa, Guannan Wei, Yangruibo Ding, Zhuo Zhang
arXiv:2607. 20759v1 Announce Type: cross Abstract: AI coding agents powered by LLMs are increasingly integrated into real-world software development, where they generate, edit, and execute code with autonomous access to local files and tools.
By Ankur Singh, Jinqiu Yang, Tse-Hsun Chen
arXiv:2606. 15762v1 Announce Type: cross Abstract: We ran 300 repeated vulnerability-finding scans to measure how repeatable agentic large language model (LLM) security review is on the same JavaScript code, prompt, and benchmark harness.
By Liran Tal, Johannes Kloos, Arsenii Rudich, Stephen Thoemmes, Manoj Nair
arXiv:2605. 09504v2 Announce Type: replace-cross Abstract: We present swarm-attack, an open-source adversarial testing framework in which multiple lightweight LLM agents coordinate through shared memory, parallel exploration, and evolutionary optimization.
By Michael A. Riegler, Inga Str\"umke
arXiv:2610.17020v1 Announce Type: cross
Abstract: Cybersecurity evaluations of AI agents have focused largely on offense, yet a working exploit does not show that an agent can make software safe. An...
By Gonzalo Gonzalez-Pumariega, Ajay Jagannath, Adit Jain, Guram Gogia, Soumyadeep Bakshi, Nazneen Rajani