arXiv AI By Oliver Stevanovic, Jasmin Wachter

Automating Attack Graph Construction for Agentic Pentesting. Towards Neuro-Symbolic Vulnerability Hunting

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv AI
Oct 27

EntailLLM: Verifying Agent-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programming

EntailLLM is a framework that verifies vulnerability discovery paths generated by large language model agents using a separate knowledge graph and logic programming. The system checks whether the agent’s traversal of a function call graph is entailed by domain knowledge, improving pooled entailment from 77% to 98% across various binaries and CWE classes. It is deployed on real medical-device binaries, achieving high entailment without per-device tuning and offering logical, explainable verification grounded in formal semantics.

By Kaustuv Mukherji, Jaikrishna Manojkumar Patil, Colton Payne, Paulo Shakarian, Dana Warmsley, Nigel Stepp, Evelyn Kim
arXiv AI
Nov 6

AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation

The paper examines how Large Language Models (LLMs) have introduced a trustworthiness crisis in cybersecurity vulnerability assessment, producing artifacts known as "AI slop" such as hallucinated vulnerabilities and incorrect patches. It presents a taxonomy of these failures, attributes them to a mismatch between deductive reasoning and probabilistic generation, and introduces the Deductive Coverage Score as a measurable proxy. The authors review mitigation strategies, critique passive approaches, and propose active neuro‑symbolic verification, concluding with new evaluation tools—CVE‑Bench and Slop‑Score—to shift focus from linguistic fluency to mathematical verifiability.

By Junchen Ding, Jialiang Dong, Yichen Zhu, Yi Liu, Gelei Deng, Willy Susilo, Siqi Ma, Yuekang Li
arXiv AI
6d ago

Specification-based Security Evaluation of Real-World Agents Tool-Calling under Long-Horizon Context Tasks

The paper introduces RAS‑Eval, a specification‑based security evaluation framework for large language model agents that uses Linear Temporal Logic (LTL) to define utility and security objectives over tool‑call trajectories and dynamic environment observations. Unlike traditional LLM‑as‑a‑Judge or Agent‑as‑a‑Judge approaches, RAS‑Eval derives deterministic verdicts via a model checker, reducing evaluator uncertainty and enabling reliable assessment over long‑horizon tasks. The authors also present a multi‑agent dataset pipeline and a benchmark of 200 normal and 800 attack tasks covering indirect prompt injection, tool poisoning, skill code injection, and tool code injection. whyItMatters":"RAS‑Eval addresses key limitations of existing security evaluations—such as evaluator hallucination and missed transient violations—by providing deterministic, environment‑aware verdicts that are crucial for safely deploying LLM agents in unattended, long‑horizon scenarios."

By Yuchuan Fu, Minghe Wang, Xiaohan Yuan, Chenlin Zou, Kaitong Jia, Jinfeng Li, Dongxia Wang