arXiv AI June 16, 2026 By Rupam Patir, Keyan Guo, Suvadra Barua, Abhijeet Pathak, Dinesh Gudimetla, Jiawei Guo, Hongxin Hu, Haipeng Cai

DualGauge: Automated Joint Security-Functionality Benchmarking of Specification-Only Code Generation by LLMs and Coding Agents

Read the original on arXiv AI →

arXiv:2511. 20709v2 Announce Type: replace-cross Abstract: Large language models (LLMs) and LLM-based coding agents are now used to generate code from natural-language specifications, yet ensuring such code is both functionally correct and secure remains a challenge.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jun 9

SecureVibeBench: Benchmarking Secure Vibe Coding of AI Agents via Reconstructing Vulnerability-Introducing Scenarios

arXiv:2509. 22097v5 Announce Type: replace-cross Abstract: Large language model-powered code agents are rapidly transforming software engineering, yet the security risks of their generated code have become a critical concern.

By Junkai Chen, Huihui Huang, Yunbo Lyu, Junwen An, Jieke Shi, Chengran Yang, Ting Zhang, Haoye Tian, Yikun Li, Zhenhao Li, Xin Zhou, Xing Hu, David Lo

llms agents benchmarks

arXiv Machine Learning

Jul 1

An Empirical Study of Security Calibration in Large Language Models for Code

arXiv:2606. 31159v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly transforming software development, yet their use in security-critical contexts raises a key question: do models know when their generated code is insecure?

By Mohammed Latif Siddiq, Md. Nafiu Rahman, Joanna C. S. Santos

llms benchmarks

arXiv Machine Learning

Jul 10

Functional and Secure Code Generation with Task Vectors

arXiv:2607. 07881v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for code generation, but they struggle to generate functional code free of security vulnerabilities.

By Felix Wang, Anudeep Das, Mei Nagappan, N. Asokan

llms benchmarks safety

arXiv AI

Jun 16

SPARK: Security Knowledge Priming and Representation-Guided Knowledge Activation for LLM-based Secure Code Generation

arXiv:2606. 16244v1 Announce Type: cross Abstract: Large language models routinely generate code with exploitable security flaws.

By Xiaoyun Xu, Lichao Wu, Jona te Lintelo, Siyu Zhang, Stjepan Picek

llms rag fine-tuning benchmarks safety data-engineering

arXiv AI

6d ago

Program Semantic Inequivalence Game with Large Language Models

arXiv:2505. 03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-trivial reasoning about program semantics.

By Antonio Valerio Miceli-Barone, Vaishak Belle, Ali Payani

llms agents fine-tuning benchmarks

arXiv AI

Aug 5

AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection

arXiv:2601. 19138v2 Announce Type: replace-cross Abstract: Secure code review is critical during pre-integration, where Atlassian developers rely on lightweight analysis tools, while deep security assessment is deferred to later stages, delaying feedback and increasing remediation costs.

By Wachiraphan Charoenwet, Kla Tantithamthavorn, Patanamon Thongtanunam, Hong Yi Lin, Minwoo Jeong, Ming Wu

llms agents

Related stories

SecureVibeBench: Benchmarking Secure Vibe Coding of AI Agents via Reconstructing Vulnerability-Introducing Scenarios

An Empirical Study of Security Calibration in Large Language Models for Code

Functional and Secure Code Generation with Task Vectors

SPARK: Security Knowledge Priming and Representation-Guided Knowledge Activation for LLM-based Secure Code Generation

Program Semantic Inequivalence Game with Large Language Models

AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection