Automating Attack Graph Construction for Agentic Pentesting. Towards Neuro-Symbolic Vulnerability Hunting
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
The Flow has not summarised this story yet — read it at arXiv AI.
EntailLLM is a framework that verifies vulnerability discovery paths generated by large language model agents using a separate knowledge graph and logic programming. The system checks whether the agent’s traversal of a function call graph is entailed by domain knowledge, improving pooled entailment from 77% to 98% across various binaries and CWE classes. It is deployed on real medical-device binaries, achieving high entailment without per-device tuning and offering logical, explainable verification grounded in formal semantics.
The paper examines how Large Language Models (LLMs) have introduced a trustworthiness crisis in cybersecurity vulnerability assessment, producing artifacts known as "AI slop" such as hallucinated vulnerabilities and incorrect patches. It presents a taxonomy of these failures, attributes them to a mismatch between deductive reasoning and probabilistic generation, and introduces the Deductive Coverage Score as a measurable proxy. The authors review mitigation strategies, critique passive approaches, and propose active neuro‑symbolic verification, concluding with new evaluation tools—CVE‑Bench and Slop‑Score—to shift focus from linguistic fluency to mathematical verifiability.
arXiv:2605. 10834v2 Announce Type: replace Abstract: AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will perform best in real-world targets.
The paper introduces RAS‑Eval, a specification‑based security evaluation framework for large language model agents that uses Linear Temporal Logic (LTL) to define utility and security objectives over tool‑call trajectories and dynamic environment observations. Unlike traditional LLM‑as‑a‑Judge or Agent‑as‑a‑Judge approaches, RAS‑Eval derives deterministic verdicts via a model checker, reducing evaluator uncertainty and enabling reliable assessment over long‑horizon tasks. The authors also present a multi‑agent dataset pipeline and a benchmark of 200 normal and 800 attack tasks covering indirect prompt injection, tool poisoning, skill code injection, and tool code injection. whyItMatters":"RAS‑Eval addresses key limitations of existing security evaluations—such as evaluator hallucination and missed transient violations—by providing deterministic, environment‑aware verdicts that are crucial for safely deploying LLM agents in unattended, long‑horizon scenarios."
arXiv:2606. 03453v1 Announce Type: cross Abstract: Vulnerability disclosure volumes now far exceed organizational assessment capacity, yet three adjacent research communities (proof-of-concept generation, vulnerability prioritization, and detection rule engineering) operate largely in isolation.
arXiv:2604.10800v2 Announce Type: replace-cross Abstract: Learned classifiers deployed in agentic pipelines face a fundamental reliability problem: predictions are probabilistic inferences, not verif...