The paper introduces CASE-Bench, a benchmark of 145 repository-level tasks derived from real CVEs across 51 projects, each run in Docker with functional tests and proof‑of‑concept exploits. It evaluates 13 context‑construction methods and six controls across six LLMs, showing that functionality‑oriented context improves functional pass rates by up to 8.2 pp but only 3.7 pp for security, that omitting safeguards can drop secure pass rates by 8.1 pp, and that coding agents see relevant safeguards in only 27‑36 % of trajectories. These results highlight how repository context shapes the security of LLM‑generated code and guide the design of security‑aware context construction and evaluation.
Large language model (LLM) coding assistants and agents construct a repository context before generating code. For security-sensitive code, this context decides whether the model sees the project's sa...
The paper introduces the Static‑Pass Dynamic‑Fail (SPDF) phenomenon, showing that static analysis can miss vulnerabilities that are exploitable at runtime. Using a three‑stage pipeline—static scanning, LLM‑driven CWE reasoning, and autonomous exploit verification—it evaluated 1,355 Python samples and found that 14.53% of samples that passed static checks were actually exploitable. The study highlights that static‑analysis success and runtime security are distinct assurance layers, especially for AI‑generated and security‑sensitive code.
The paper introduces AgentSecGraph, a static analysis framework that builds a Security-Aware Agent Dependency Graph (Security-ADG) for each security-sensitive operation in large language model (LLM) agents. It enriches operation identity with agent relevance, source and dependency evidence, trust-boundary context, guard evidence, and external-effect semantics. The authors also present AgentSecBench, a corpus of 67 real-world LLM-agent repositories, and demonstrate that their analyzer identifies thousands of operation candidates, recovers substantial dependency and guard evidence, and distinguishes guarded behaviors from vulnerabilities with high accuracy.
arXiv:2603.18740v3 Announce Type: replace-cross Abstract: Automated Code Review (ACR) systems integrating Large Language Models (LLMs) are increasingly adopted in software development workflows, rang...
arXiv:2407.02395v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for program synthesis, yet they often generate code that is functionally plausible but ins...
arXiv:2509. 22097v5 Announce Type: replace-cross Abstract: Large language model-powered code agents are rapidly transforming software engineering, yet the security risks of their generated code have become a critical concern.
arXiv:2606. 31159v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly transforming software development, yet their use in security-critical contexts raises a key question: do models know when their generated code is insecure?
The paper introduces CodePoisonRAG, a framework that poisons retrieval-augmented code generation systems by transforming benign code artifacts into malicious ones. It injects CWE-specific vulnerabilities and false safety claims into a single task-matched artifact, achieving high success rates across multiple generators and even against a defense system. The study demonstrates that attackers can target and propagate specific weaknesses without altering the underlying language model.
The paper introduces RedHerring, a defense mechanism that inserts safe decoy vulnerabilities into code repositories to divert autonomous LLM agents’ verification efforts away from real security flaws. By embedding CVE-derived vulnerability chains with false bridges and providing a private certificate for quick verification, RedHerring forces agents to spend a significant portion of their limited resources on decoys. Experiments on 33 OSS‑Fuzz projects show a 38.7‑60.4% reduction in discovered real vulnerabilities, even when agents are aware of decoys.
Large Language Models (LLMs) are evolving from simple code completion tools to repository‑scale agents capable of retrieving context, editing files, executing tools, and engaging in security‑sensitive workflows. A structured survey up to May 31 2026 reviews LLM work across software engineering and security tasks, adaptation mechanisms, artifact granularity, and evaluation design, and introduces an assurance framework that separates functional correctness, security, operational reliability, evidence provenance, and agent authority. The review highlights that while execution feedback and repository access improve engineering task completion, they do not guarantee security, and static‑analysis labels rarely ensure deployable correctness; it also identifies common validity threats and proposes a minimum reporting protocol and a research agenda focused on jointly secure‑and‑functional benchmarks, repository‑scale threat models, calibrated human oversight, longitudinal maintainability evidence, and reproducible agent evaluation.
arXiv:2511. 20709v2 Announce Type: replace-cross Abstract: Large language models (LLMs) and LLM-based coding agents are now used to generate code from natural-language specifications, yet ensuring such code is both functionally correct and secure remains a challenge.
arXiv:2607. 01916v1 Announce Type: new Abstract: Large language model agents can repair real repository issues, but they often spend large context budgets on whole-file reads, broad searches, and long terminal outputs where useful evidence is mixed with irrelevant code and logs.