arXiv Machine Learning
Jul 17
Agentic Vulnerability Reasoning on COTS Binaries
arXiv:2605. 05000v2 Announce Type: replace-cross Abstract: LLM agents have been increasingly adopted for solving security tasks.
Arbiter: Detecting Interference in LLM Agent System Prompts
Arbiter is a framework that uses formal evaluation rules and multi-model LLM scouring to detect interference patterns in system prompts for large language model (LLM) coding agents. Applied to Claude Code, Codex CLI, and Gemini CLI, it uncovered 152 findings in an undirected phase and 21 hand‑labeled interference patterns in a directed analysis. The study shows that prompt architecture correlates with failure class, multi-model evaluation reveals different vulnerability classes than single-model analysis, and a structural data loss issue in Gemini CLI was identified before Google’s patch addressed only the symptom.
Related stories
arXiv AI
Aug 18
Evaluating Agentic Code Repair Capabilities in Distributed Systems
arXiv:2608. 14863v1 Announce Type: cross Abstract: LLM-based coding agents have advanced rapidly on single-process SWE tasks, with frontier models now clustering in the high-70s on SWE-bench Verified.
arXiv Machine Learning
Sep 10
VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities
arXiv:2609.08040v1 Announce Type: cross Abstract: The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing d...
arXiv Machine Learning
Sep 2
AgentProv: Auditing Agentic LLM API Providers via Tool-use Policy Probes
arXiv:2609.00052v1 Announce Type: cross Abstract: Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped, for example to...
arXiv AI
Jun 29
Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
arXiv:2601. 21233v2 Announce Type: replace Abstract: Autonomous code agents built on large language models are reshaping software and AI development through tool use, long-horizon reasoning, and self-directed interaction.
arXiv AI
Jun 18
Code-Augur: Agentic Vulnerability Detection via Specification Inference
arXiv:2606. 18619v1 Announce Type: cross Abstract: The advent of agentic vulnerability detection is already becoming a watershed moment for software security.
arXiv AI
Oct 7
Learning from Failures: A Failure-Driven Prompt Refinement for LLM-Based Vulnerability Analysis
arXiv:2610.08405v1 Announce Type: cross Abstract: Large Language Models have emerged as promising tools for software vulnerability analysis, but their effectiveness depends heavily on prompt design....
arXiv AI
Jul 13
The Patchwork Problem in LLM-Generated Code
arXiv:2607. 08981v1 Announce Type: cross Abstract: LLM-generated code often compiles, passes tests, and appears correct, yet breaks once deployed.
arXiv AI
Jul 3
Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification
arXiv:2607. 01793v1 Announce Type: new Abstract: LLM agents increasingly perform autonomous actions through external tools, leading to complex and evolving safety risks.
arXiv AI
Jun 18
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
arXiv:2606. 18356v1 Announce Type: cross Abstract: Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects.
arXiv AI
Oct 27
Can Agentic Code Review Catch Vulnerabilities Before They Are Merged?
The paper introduces SecPRBench, a benchmark that evaluates large language model agents on real-world bug‑introducing commits from critical software systems such as Firefox, V8, and OpenSSL. It automates harvesting, building, and validating these commits, and uses an LLM‑as‑a‑judge to match findings with ground‑truth vulnerabilities. Across eight state‑of‑the‑art models, recall ranges from 32.5% to 78.1%, with Grok 4.6 and GPT‑6 sol performing best, while security‑focused harnesses like OpenAI’s Codex Security and Anthropic’s Defending Code influence recall differently.
arXiv AI
Aug 3
AgenticRepair: Multi-Faceted Program Context Engineering for Agentic Vulnerability Repair
arXiv:2607. 29422v1 Announce Type: cross Abstract: Automated vulnerability repair aims to reduce the time and effort required to patch security flaws from a vulnerability triage report.