← Back to all news
arXiv AI By Chenyuan Yang, Hwiwon Lee, Jongseong Kim, Yihan Yang, Dongjun Kim, Yinfang Chen, Lingming Zhang

CyberDual: AI Attackers versus AI Code Reviewers in Real Software Maintenance

Read the original on arXiv AI →

CyberDual is a competitive framework where AI agents act as both attackers and reviewers in real software maintenance tasks. Attackers embed exploitable bugs into open‑source projects, while reviewer agents attempt to detect and fix them; success is measured by re‑executing the attacker’s proof‑of‑concept on the reviewed code. Across 1,088 valid attacks, three reviewers caught 85‑91% of bugs, yet about one in six attacks still produced a working exploit after review, highlighting gaps in current AI code‑review tools.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 27

Can Agentic Code Review Catch Vulnerabilities Before They Are Merged?

The paper introduces SecPRBench, a benchmark that evaluates large language model agents on real-world bug‑introducing commits from critical software systems such as Firefox, V8, and OpenSSL. It automates harvesting, building, and validating these commits, and uses an LLM‑as‑a‑judge to match findings with ground‑truth vulnerabilities. Across eight state‑of‑the‑art models, recall ranges from 32.5% to 78.1%, with Grok 4.6 and GPT‑6 sol performing best, while security‑focused harnesses like OpenAI’s Codex Security and Anthropic’s Defending Code influence recall differently.

By Yihan Yang, Chenyuan Yang, Zijie Zhao, Anh Nguyen Hoang, Ziqi Zhang, Weidong Wang, Zhiheng Lyu, Lingming Zhang
llmsagentsbenchmarks
More like this →
arXiv AI
Oct 2

Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents

The paper investigates whether reviewers with limited expertise can reliably determine if a coding agent’s patch resolves a bug, using 411 execution-labeled traces from three agents and 101 controlled cases. It finds that providing structured but unchecked evidence can improve defect detection and reduce over-rejection, and that a frozen cascade of static errors and generated tests achieves high coverage and catch rates on held‑out GPT‑5.4 and Gemini traces. Reviewer size alone does not predict quality, and the study highlights the challenge of creating reliable checks without official tests.

By Junyu Guo, Shangding Gu, Ming Jin, Javad Lavaei
llmsagents
More like this →