arXiv:2606. 18619v1 Announce Type: cross Abstract: The advent of agentic vulnerability detection is already becoming a watershed moment for software security.
By Zhengxiong Luo, Mehtab Zafar, Dylan Wolff, Abhik Roychoudhury
Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first introduce vulnerabilities into codebases. VICs are essential for determining the full range of vulnerable software versions.
arXiv:2608. 12246v1 Announce Type: cross Abstract: Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first introduce vulnerabilities into codebases.
By Jin Lu, Xuening Han, Yang Zhong, Lin Tan, Kevin Luo, Andrew Gacek, Neha Rungta
The paper introduces SUSVIBES, a benchmark of 186 real‑world software engineering tasks where human programmers have committed vulnerable code. It evaluates 12 popular coding‑agent settings on these tasks and finds that all agents perform poorly in terms of security, with only 11.8% of solutions from SWE‑Agent with Claude 4 Sonnet being secure despite 57% being functionally correct. Attempts to mitigate security issues by adding vulnerability hints to the prompts do not improve results.
By Songwen Zhao, Danqing Wang, Kexun Zhang, Jiaxuan Luo, Zhuo Li, Lei Li
arXiv:2407.02395v3 Announce Type: replace-cross
Abstract: Large language models (LLMs) are increasingly used for program synthesis, yet they often generate code that is functionally plausible but ins...
By Jiexin Wang, Liuwen Cao, Xitong Luo, Yang Cao, Zhenghao Li, Yunyi Xiao, Mengchen Zhao, Adam Jatowt, Yi Cai
arXiv:2606. 19149v1 Announce Type: cross Abstract: Automated vulnerability discovery in large codebases remains challenging: traditional static analysis produces high false-positive rates, while dynamic approaches such as fuzzing require substantial infrastructure and often target narrow classes of bugs.
By Nahum Korda, Gadi Evron