Large Language Models (LLMs) are evolving from simple code completion tools to repository‑scale agents capable of retrieving context, editing files, executing tools, and engaging in security‑sensitive workflows. A structured survey up to May 31 2026 reviews LLM work across software engineering and security tasks, adaptation mechanisms, artifact granularity, and evaluation design, and introduces an assurance framework that separates functional correctness, security, operational reliability, evidence provenance, and agent authority. The review highlights that while execution feedback and repository access improve engineering task completion, they do not guarantee security, and static‑analysis labels rarely ensure deployable correctness; it also identifies common validity threats and proposes a minimum reporting protocol and a research agenda focused on jointly secure‑and‑functional benchmarks, repository‑scale threat models, calibrated human oversight, longitudinal maintainability evidence, and reproducible agent evaluation.
arXiv:2607. 05842v1 Announce Type: cross Abstract: Large language model (LLM)-assisted software security operates at a difficult boundary: the vulnerability-analysis terminology needed for legitimate code review, triage, and repair can closely resemble terminology associated with misuse.
The paper introduces AgentSecGraph, a static analysis framework that builds a Security-Aware Agent Dependency Graph (Security-ADG) for each security-sensitive operation in large language model (LLM) agents. It enriches operation identity with agent relevance, source and dependency evidence, trust-boundary context, guard evidence, and external-effect semantics. The authors also present AgentSecBench, a corpus of 67 real-world LLM-agent repositories, and demonstrate that their analyzer identifies thousands of operation candidates, recovers substantial dependency and guard evidence, and distinguishes guarded behaviors from vulnerabilities with high accuracy.
SWE-Prometheus is a new benchmark that evaluates large language model coding agents on the broader task of improving repository engineering governance, rather than just fixing individual issues. It presents fixed snapshots with open-ended objectives, requiring agents to identify risks, prioritize interventions, and verify changes across six governance dimensions. The benchmark includes 60 repositories and reports metrics such as Normalized Governance Improvement and behavior‑breakage rates, providing a nuanced view of how different models affect governance artifacts and execution‑backed improvements.
arXiv:2407.02395v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for program synthesis, yet they often generate code that is functionally plausible but ins...
arXiv:2606. 18356v1 Announce Type: cross Abstract: Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects.
The study investigates how software developers, architects, and AI practitioners select and integrate Large Language Models (LLMs) into modern software systems. Interviews with 22 professionals reveal that functional criteria—such as performance, accuracy, cost, and specific features—dominate model choice, while security concerns are rarely considered. The research highlights a pervasive neglect of established software supply‑chain security lessons, leading to vulnerabilities like malicious components, data leakage, and unintended behavior, and offers actionable recommendations for a proactive, security‑by‑design approach.
arXiv:2606. 31639v1 Announce Type: cross Abstract: Large language models are no longer only text generators.
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It evaluates candidate skills against source evidence and nine safety properties, then tests them in a controlled environment to capture execution traces and identify failures. The system iteratively refines skills based on these results, achieving high precision in vulnerability detection and significantly improving task performance and security rates.
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It first checks functional claims against evidence and evaluates artifacts against nine safety properties, then tests admitted skills in a controlled environment to capture execution traces and identify failures. The approach achieves perfect precision and recall in vulnerability detection, significantly reduces attack success rates, and boosts task effectiveness and security rates in skill generation benchmarks.
SpecCloak investigates how presenting coding tasks as engineering specifications—via the PseudoSpec template—affects large language models’ refusal to generate harmful code. Experiments on five open-weight models show that specification prompting significantly lowers refusal rates even when risk keywords remain, shifting internal representations toward code-output contexts. Building on this, SpecCloak combines PseudoSpec with discriminator-guided optimization to achieve a 28.91% attack success rate, while SpecReveal offers a defense that detects such attacks with 96.88% accuracy without fine‑tuning the target model.
The paper "Beyond Reproducibility: Towards Security-Aware Evaluation of Research Artifacts" examines 1,388 research artifacts from top security conferences, uncovering 132,431 candidate security findings through static analysis. It introduces a taxonomy for context-aware security assessment and presents SAFE, an autonomous framework that accurately distinguishes security-relevant findings and classifies risk types. The study demonstrates that nearly 45% of findings are security-relevant, highlighting the need for security-aware evaluation alongside traditional reproducibility checks.
The paper introduces a deterministic AI security risk assessment framework that transforms diverse engineering artefacts into a standardized Control ID taxonomy scored on a four‑level ordinal scale. It compiles technique‑level predicates from a fixed MITRE ATLAS snapshot, linking each control to mitigation and producing traceable feasibility and impact outputs. The framework is formally verified for boundedness, totality, consistency, and monotonicity, and is evaluated on five open‑source AI projects, showing that strengthened controls lower feasibility scores while residual risks persist when core controls are missing.