arXiv AI September 17, 2026 By Rushabh Vipulkumar Patel, Dipo Dunsin, Mohammed Almaiah, Mohamed Chahine Ghanem

PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs

PentestChain is a ten‑phase automated penetration testing framework that uses a cost‑aware AI cascade, starting with a local 7B‑parameter Ollama model (qwen2.5‑7b) and then free‑tier OpenRouter and Cerebras models, with a rule‑based fallback. It exposes the entire pipeline through a Model Context Protocol (MCP) server that includes eleven tools. The authors evaluate the framework using standard testbeds (AutoPenBench, Cybench subset, PentestGPT 182‑sub‑task benchmark) and report that the local model keeps paid‑API cost at zero while detecting 26 services and enriching 34 CVEs on legacy targets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Sep 16

PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs

AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement token cost makes continuous, automated testing unaffordable for the smaller organ...

llms agents benchmarks

arXiv AI

Jul 16

Baselines Before Architecture: Evaluating Coding Agents for Autonomous Penetration Testing

arXiv:2607. 13085v1 Announce Type: cross Abstract: Recent autonomous penetration testing papers report high benchmark scores while adding multi-component security harnesses around frontier LLMs.

By Ananda Dhakal, Krish Neupane, Aarjan Chaudhary

llms agents benchmarks

arXiv AI

Jun 11

Are Frontier LLMs Ready for Cybersecurity? Evidence for Vertical Foundation Models from Dual-Mode Vulnerability Benchmarks

arXiv:2605. 23243v2 Announce Type: replace-cross Abstract: We evaluate whether frontier LLMs are ready for cybersecurity through a dual-mode benchmark: white-box function-level vulnerability detection (VulnLLM-R, across C/Java/Python) and black-box web application security testing (five production-style applications with 118 ground-truth vulnerabilities across 20+ CWE families, which we will open-source).

By Vivek Dahiya, Sunny Nehra, Vipul Dholariya, Bhavik Shangari, Chandra Khatri

llms agents reinforcement-learning benchmarks

arXiv AI

Sep 23

APIOT: Autonomous Vulnerability Management Across Bare-Metal Industrial OT Networks

arXiv:2605.02346v2 Announce Type: replace-cross Abstract: Operational technology (OT) devices run safety-critical physical processes, yet their security testing remains manual and expert-intensive. A...

By Adel ElZemity, Budi Arief, Shujun Li, George Oikonomou, James Pope

llms agents safety

arXiv AI

Jul 15

From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World

arXiv:2605. 10834v2 Announce Type: replace Abstract: AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will perform best in real-world targets.

By Pedro Conde, Henrique Branquinho, Valerio Mazzone, Bruno Mendes, Andr\'e Baptista, Nuno Moniz

llms agents benchmarks

arXiv AI

Sep 23

From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation

arXiv:2609.22664v1 Announce Type: cross Abstract: Research on large language model agents for penetration testing is evaluated almost entirely by capability: whether the agent captures a flag or repr...

By Joas Antonio dos Santos Barbosa

llms agents benchmarks safety

One email a morning, machine-written

Related stories

PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs

Baselines Before Architecture: Evaluating Coding Agents for Autonomous Penetration Testing

Are Frontier LLMs Ready for Cybersecurity? Evidence for Vertical Foundation Models from Dual-Mode Vulnerability Benchmarks

APIOT: Autonomous Vulnerability Management Across Bare-Metal Industrial OT Networks

From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World

From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation