← Back to all news
arXiv AI By Ira Ceka, Arnav Garg, Ryan Lutz, Ting Zhang, Ratnadira Widyasari, Gail Kaiser, Baishakhi Ray

MetaVul: Agentic Harness Discovery for Vulnerability Detection

Read the original on arXiv AI →

MetaVul is a self‑improving framework that discovers and refines harnesses—code scaffolding around large language model agents—for vulnerability detection. Starting from a seed reasoning scaffold, it iteratively optimizes harnesses using feedback from paired vulnerable and patched functions, producing Pareto‑optimal designs that can be reused across different models. Experiments show that optimized harnesses can boost pairwise vulnerability detection by up to 25.8 percentage points and transfer effectively between models without further tuning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 7

Direction for Detection: A Survey of Automated Vulnerability Detection and all of its Pain Points

The paper surveys 87 influential studies on machine‑learning‑based automated vulnerability detection (ML4AVD), categorizing them by problem formulation, input and detection granularity, target languages, evaluation metrics, datasets, and detection approaches. It identifies twelve self‑reinforcing pain points—such as overreliance on binary classification of C/C++ function‑level vulnerabilities, limited language coverage, and intertwined datasets, baselines, and metrics—that trap the field in a narrow, artificial problem space. The authors propose concrete recommendations to break these feedback loops and evaluate a recent high‑profile effort, AIxCC, against these guidelines, reflecting on ML4AVD’s relevance amid the rise of agentic AI.

By Dan Ristea, Shae McFadden, Ezzeldin Shereen, Madeleine Dwyer, Sanyam Vyas, Chris Hicks, Vasilios Mavroudis
agents
More like this →
arXiv AI
Sep 4

PatchBench: Evaluating AI Agents for Vulnerability Patching

PatchBench introduces a benchmark to evaluate AI agents on realistic vulnerability patching tasks, addressing two key threats to validity: patch memorization and surface-level fixes that merely suppress crashes. The study finds that 25% of agent patches resemble historical developer patches, and that PoC-only validation inflates success rates by 1.83× on average. PatchBench mitigates these issues by selecting vulnerabilities whose true fixes lie outside the crash stack, migrating historical vulnerabilities into new contexts, and employing rigorous validation for security and semantic correctness.

By Chihao Shen, Jiacheng Li, Aastha Mahajan, Jeffery Siyuan Tian, Yonghwi Kwon, Yizheng Chen
agentsbenchmarks
More like this →