The paper introduces SUSVIBES, a benchmark of 186 real‑world software engineering tasks where human programmers have committed vulnerable code. It evaluates 12 popular coding‑agent settings on these tasks and finds that all agents perform poorly in terms of security, with only 11.8% of solutions from SWE‑Agent with Claude 4 Sonnet being secure despite 57% being functionally correct. Attempts to mitigate security issues by adding vulnerability hints to the prompts do not improve results.
By Songwen Zhao, Danqing Wang, Kexun Zhang, Jiaxuan Luo, Zhuo Li, Lei Li
arXiv:2607. 23088v1 Announce Type: cross Abstract: Large Language Models (LLMs) are widely used for code generation, yet their security behavior in realistic development workflows remains underexplored.
By Lixun Ma, Ruolong Ma, Bei Wang, Feng Wei, Zhenguang Liu, Lorenzo Cavallaro, Wentao Chen
arXiv:2610.14766v1 Announce Type: cross
Abstract: AI coding agents now write much of the code that ships, yet whether that code is secure remains unclear. We introduce the Last Secure Code Benchmark,...
By Tianyi Wu, Chao Ge, Peng Chen, Fengji Zhang, Yongxing Wu, Yuchong Xie, Jian Li, Chaoyuan Peng, Kaikai Zhang, Dong Huang, Mingzhe Du, Yukai Huang, Jiaheng Zhang, Terry Yue Zhuo, See-Kiong Ng
arXiv:2607. 23710v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their ability to autonomously generate secure authentication code remains uncertain.
By Ishpuneet Singh, Shreyas Mahajan, Gurjot Singh, Maninder Singh
SecureVibe is a training recipe designed to enhance the security of vibe coding by explicitly targeting planning and testing for code security. It combines supervised fine‑tuning on a security suite with post‑training methods (SECUREVIBE_rl and SECUREVIBE_hg) that use verifiable execution feedback and hint‑based self‑supervision. The approach outperforms baselines on multiple security coding benchmarks, improving security pass@1 by up to 6.9 points on BaxBench and 11.5 points on SusVibes, while also boosting functionality pass@1 on both security and generic coding tasks.
By Danqing Wang, Baolin Peng, Zhepei Wei, Isadora White, Wenlin Yao, Hao Cheng, Qianhui Wu, Minseon Kim, Xingdi Yuan, Lei Li, Jianfeng Gao
arXiv:2601. 19138v2 Announce Type: replace-cross Abstract: Secure code review is critical during pre-integration, where Atlassian developers rely on lightweight analysis tools, while deep security assessment is deferred to later stages, delaying feedback and increasing remediation costs.
By Wachiraphan Charoenwet, Kla Tantithamthavorn, Patanamon Thongtanunam, Hong Yi Lin, Minwoo Jeong, Ming Wu