arXiv AI August 18, 2026 By Zhiyu Zhang, Tingyue Wen, Senke Sun, Dengxiang Liang, Enhao Huang

WeSCE: A Benchmark for Measuring Security Drift in LLM-Driven Code Editing

arXiv:2608. 15092v1 Announce Type: cross Abstract: In this work, we introduce WeSCE, a benchmark for quantifying security drift in code editing under weak-security constraints, where tasks specify only functional objectives without explicit security requirements.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 28

Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios

arXiv:2607. 23088v1 Announce Type: cross Abstract: Large Language Models (LLMs) are widely used for code generation, yet their security behavior in realistic development workflows remains underexplored.

By Lixun Ma, Ruolong Ma, Bei Wang, Feng Wei, Zhenguang Liu, Lorenzo Cavallaro, Wentao Chen

llms benchmarks

arXiv Computation and Language

Sep 15

Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarking

arXiv:2407.02395v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for program synthesis, yet they often generate code that is functionally plausible but ins...

By Jiexin Wang, Liuwen Cao, Xitong Luo, Yang Cao, Zhenghao Li, Yunyi Xiao, Mengchen Zhao, Adam Jatowt, Yi Cai

llms agents benchmarks

arXiv Machine Learning

Jul 1

An Empirical Study of Security Calibration in Large Language Models for Code

arXiv:2606. 31159v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly transforming software development, yet their use in security-critical contexts raises a key question: do models know when their generated code is insecure?

By Mohammed Latif Siddiq, Md. Nafiu Rahman, Joanna C. S. Santos

llms benchmarks

arXiv AI

Sep 25

Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning

The paper introduces CodeScan, a black-box, vulnerability-oriented scanning framework designed to detect data poisoning and backdoor attacks in code generation large language models (LLMs). CodeScan operates by analyzing structural similarities across multiple code generations, normalizing them with abstract syntax tree (AST) techniques, and then applying LLM-based vulnerability analysis to identify recurring insecure patterns. Evaluations on 117 models across three architectures and multiple sizes show over 97% detection accuracy with fewer false positives compared to prior methods.

By Shenao Yan, Shan Jin, Shimaa Ahmed, Sunpreet Singh Arora, Yiwei Cai, Yizhen Wang, Yuan Hong

llms

arXiv AI

Jun 8

Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software

arXiv:2602. 04894v4 Announce Type: replace-cross Abstract: LLMs are increasingly used for code generation, but their outputs often follow recurring templates that can induce predictable vulnerabilities.

By Tomer Kordonsky, Amit LeVi, Maayan Yamin, Noam Benzimra, Avi Mendelson

llms benchmarks

arXiv Machine Learning

Aug 6

A Comprehensive Evaluation of Code Language Models for Security Patch Detection

arXiv:2605. 13138v2 Announce Type: replace-cross Abstract: Automated detection of vulnerability-fixing commits (\vfcs) is critical for timely security patch deployment, as advisory databases lag patch releases by a median of 25 days and many fixes never receive advisories.

By Nils Loose, Joseph Bienh\"uls, Kristoffer Hempel, Felix M\"achtle, Thomas Eisenbarth

llms fine-tuning data-engineering

One email a morning, machine-written

Related stories

Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios

Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarking

An Empirical Study of Security Calibration in Large Language Models for Code

Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning

Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software

A Comprehensive Evaluation of Code Language Models for Security Patch Detection