arXiv:2606. 08960v1 Announce Type: cross Abstract: Agent benchmarks score submissions with outcome verifiers that are typically hand-written and brittle, leaving them open to reward hacking.
arXiv:2610.17020v1 Announce Type: cross Abstract: Cybersecurity evaluations of AI agents have focused largely on offense, yet a working exploit does not show that an agent can make software safe. An...
arXiv:2607. 11698v1 Announce Type: cross Abstract: Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable.
AgentXploit is a two‑role auditing system that separates repository‑level attack‑path discovery from runtime exploitation for AI agents. The Analyzer Agent traces attacker‑controlled inputs to sensitive operations and records candidate attack paths, while the Exploiter Agent turns these paths into concrete attacks and refines them using runtime feedback. The system is evaluated on AgentXploit‑Bench, a benchmark of 72 reproducible vulnerabilities across 12 open‑source AI‑agent systems, achieving 59.3% end‑to‑end success compared to 38.4% for Codex, and 79.2% attack success on AgentDojo versus 52.7% for AgentVigil.
Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable. Red-teaming must therefore keep pace with evolving models and tools.
arXiv:2608. 06471v1 Announce Type: cross Abstract: Despite recent advances, frontier large language model (LLM) agents remain limited in discovering and patching complex vulnerabilities in real-world software.
arXiv:2606. 30755v1 Announce Type: cross Abstract: Claw-like AI agents (e.
arXiv:2609.08040v1 Announce Type: cross Abstract: The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing d...
AutoRedClaw is an automated red‑teaming framework designed for claw‑style agents such as OpenClaw, which maintain persistent configuration, memory, and reusable skills. It generates end‑to‑end attacks and the necessary environments, using a strict quality‑checking protocol and execution‑based validation to ensure realistic, effective samples. The framework produces a 330‑case benchmark, RedClaw‑Bench, achieving over 30% effective attack rate at less than $0.10 per successful sample and demonstrating higher attack success rates than existing benchmarks across multiple agent systems.
arXiv:2606. 12797v1 Announce Type: new Abstract: Agentic large language model systems that autonomously invoke tools, maintain persistent memory, and execute multi-step plans are increasingly deployed in public-facing domains, including government services, healthcare triage, and financial advising.
arXiv:2606. 18619v1 Announce Type: cross Abstract: The advent of agentic vulnerability detection is already becoming a watershed moment for software security.
HardEval is an execution‑verified benchmark that tests language‑model‑assisted security hardening on a live Red Hat Enterprise Linux 8 system. It uses STIG controls that fail on an unhardened baseline, and models generate Bash scripts that are run on a virtual machine and graded by OpenSCAP/OVAL compliance checks. The benchmark reveals significant differences in model performance across security domains and highlights failures where code appears plausible but does not meet the exact compliance condition.