Large Language Models (LLMs) are evolving from simple code completion tools to repository‑scale agents capable of retrieving context, editing files, executing tools, and engaging in security‑sensitive workflows. A structured survey up to May 31 2026 reviews LLM work across software engineering and security tasks, adaptation mechanisms, artifact granularity, and evaluation design, and introduces an assurance framework that separates functional correctness, security, operational reliability, evidence provenance, and agent authority. The review highlights that while execution feedback and repository access improve engineering task completion, they do not guarantee security, and static‑analysis labels rarely ensure deployable correctness; it also identifies common validity threats and proposes a minimum reporting protocol and a research agenda focused on jointly secure‑and‑functional benchmarks, repository‑scale threat models, calibrated human oversight, longitudinal maintainability evidence, and reproducible agent evaluation.
By Wei Lin, Tao Zhou, Zhaofei Xie, Changgui Hong
arXiv:2606. 31639v1 Announce Type: cross Abstract: Large language models are no longer only text generators.
By Seyed Bagher Hashemi Natanzi, Bo Tang
arXiv:2606. 18356v1 Announce Type: cross Abstract: Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects.
By Yuchuan Tian, Mengyu Zheng, Haocheng Mei, Ye Yuan, Chao Xu, Xinghao Chen, Hanting Chen, Yu Wang
arXiv:2606. 31159v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly transforming software development, yet their use in security-critical contexts raises a key question: do models know when their generated code is insecure?
By Mohammed Latif Siddiq, Md. Nafiu Rahman, Joanna C. S. Santos
arXiv:2606. 05396v1 Announce Type: cross Abstract: Producing a labeled vulnerable code at scale is a recurring obstacle for learning-based vulnerability detection: mined corpora carry substantial label noise, and existing LLM-based augmentation propagates these inaccuracies because it transforms vulnerable seeds rather than synthesising vulnerabilities from a specification.
By Cristina Carleo, Pietro Liguori, Naghmeh Ivaki, Domenico Cotroneo
arXiv:2510. 15476v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used as interfaces to information, code, and real-world services, making prompt-level security failures a practical concern.
By Hanbin Hong, Shuang Wu, Shuya Feng, Nima Naderloui, Shenao Yan, Jingyu Zhang, Ali Arastehfard, Heqing Huang, Yuan Hong
The paper introduces Porting Benchmark, a dataset of 1,234 security patch backporting cases covering cross-version, cross-branch, and cross-repository scenarios, along with a unified evaluation framework. Five tools—spanning program analysis, LLM prompting, and LLM agents—are evaluated under aligned settings, revealing that performance varies significantly across tools and patch complexity, with success rates dropping sharply for structurally complex patches. The study identifies four root-cause categories for failures and demonstrates that reference-based benchmark scores may not fully reflect real-world remediation, as executable validation uncovers additional integration issues.
arXiv:2609.15939v1 Announce Type: cross
Abstract: Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can dete...
By Aman Priyanshu, Supriti Vijay, Kimia Majd, Xuhong He, Fraser Burch, Takahiro Matsumoto, Jianliang He, Baturay Saglam, Arthur Goldblatt, Zhuoran Yang, Amin Karbasi
arXiv:2608.23181v1 Announce Type: cross
Abstract: As large language models (LLMs) continue to advance in coding capabilities, their potential in cybersecurity has drawn increasing research attention,...
By Jian Yang, Haau-Sing Li, Shawn Guo, Zixi Zhao, Yibo Tan, Jiajun Wu, Aishan Liu, Xianglong Liu, Tianyu Zheng, Bryan Dai, Chengran Yang
The paper introduces Porting Benchmark, a curated dataset of 1,234 security patch backporting cases that span cross-version, cross-branch, and cross-repository scenarios, along with a common evaluation framework. Five tools—spanning program analysis, LLM prompting, and LLM agents—are evaluated under aligned settings, revealing that performance varies significantly across tools and that complex patches (Type-IV) see a sharp drop in success rate. The study identifies four root-cause categories for failures and demonstrates that reference-based benchmark scores may not fully capture real-world remediation, as executable validation uncovers additional integration issues.
By Jincheng Yang, Yulong Fu, Chengwei Liu, Lyuye Zhang, Fangyuan Zhang, Bingyang Ren, Yang Liu, Hui Li
arXiv:2608. 10530v1 Announce Type: cross Abstract: Large Language Models (LLMs) have undergone a shift from stateless conversational interfaces to autonomous agents capable of multi-step planning, tool invocation, code execution, and maintaining persistent memory.
By Md Jafrin Hossain, Mohammad Arif Hossain, Nirwan Ansari
The paper introduces BUGSTONE‑E2E, a framework that converts vulnerability history into executable detection rules and validates them. It mines reusable rules from fixing commits, organizes them by CWE and language, and applies a funnel‑shaped pipeline that starts with lightweight analysis and culminates in LLM‑guided inspection, runtime verification, and patch generation. Using 19,325 high‑severity CVEs, the system identified 2,710 fixing commits, created 1,033 detection rules across 56 CWE families, and produced runtime evidence for 644 findings in 14 programs.
By Qiushi Wu, Kevin Eykholt, Youngja Park, Xiaokui Shu, Dhilung Kirat, Douglas Lee Schales, Ian Molloy