arXiv AI By Kaustuv Mukherji, Jaikrishna Manojkumar Patil, Colton Payne, Paulo Shakarian, Dana Warmsley, Nigel Stepp, Evelyn Kim

EntailLLM: Verifying Agent-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programming

Read the original on arXiv AI →

EntailLLM is a framework that verifies vulnerability discovery paths generated by large language model agents using a separate knowledge graph and logic programming. The system checks whether the agent’s traversal of a function call graph is entailed by domain knowledge, improving pooled entailment from 77% to 98% across various binaries and CWE classes. It is deployed on real medical-device binaries, achieving high entailment without per-device tuning and offering logical, explainable verification grounded in formal semantics.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
1d ago

AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation

The paper examines how Large Language Models (LLMs) have introduced a trustworthiness crisis in cybersecurity vulnerability assessment, producing artifacts known as "AI slop" such as hallucinated vulnerabilities and incorrect patches. It presents a taxonomy of these failures, attributes them to a mismatch between deductive reasoning and probabilistic generation, and introduces the Deductive Coverage Score as a measurable proxy. The authors review mitigation strategies, critique passive approaches, and propose active neuro‑symbolic verification, concluding with new evaluation tools—CVE‑Bench and Slop‑Score—to shift focus from linguistic fluency to mathematical verifiability.

By Junchen Ding, Jialiang Dong, Yichen Zhu, Yi Liu, Gelei Deng, Willy Susilo, Siqi Ma, Yuekang Li
arXiv AI
5d ago

MetaVul: Agentic Harness Discovery for Vulnerability Detection

MetaVul is a self‑improving framework that discovers and refines harnesses—code scaffolding around large language model agents—for vulnerability detection. Starting from a seed reasoning scaffold, it iteratively optimizes harnesses using feedback from paired vulnerable and patched functions, producing Pareto‑optimal designs that can be reused across different models. Experiments show that optimized harnesses can boost pairwise vulnerability detection by up to 25.8 percentage points and transfer effectively between models without further tuning.

By Ira Ceka, Arnav Garg, Ryan Lutz, Ting Zhang, Ratnadira Widyasari, Gail Kaiser, Baishakhi Ray
arXiv AI
Aug 24

Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems

The paper introduces Trustworthy RAG, an evaluation agent designed to detect misinformation and knowledge poisoning in Retrieval-Augmented Generation systems. It combines natural language inference verification, a five-signal poison detector, and a weighted Trust Index to assess the reliability of retrieved content. Experiments on multiple LLMs show high accuracy and precision, with the agent effectively blocking unsafe advice in a secure-coding assistant scenario.

By Balkrishna Giri, Md Toufique Hasan, Jussi Rasku, Muhammad Waseem, Pekka Abrahamsson