arXiv Machine Learning By Terry Yue Zhuo, Dingmin Wang, Hantian Ding, Varun Kumar, Zijian Wang

Cyber-Zero: Training Cybersecurity Agents without Runtime

Read the original on arXiv Machine Learning →

Cyber-Zero introduces the first runtime‑free framework for training cybersecurity agents by synthesizing high‑quality agent trajectories using publicly available CTF writeups and persona‑driven LLM simulation. The approach generates realistic, long‑horizon interaction sequences without requiring actual execution environments, enabling LLM‑based agents to achieve up to 11.4% absolute performance gains on three major CTF benchmarks. The best model, Cyber‑Zero‑32B, matches proprietary systems such as DeepSeek‑V3‑0324 and Claude‑3.5‑Sonnet while being more cost‑effective, demonstrating that runtime‑free trajectory synthesis can democratize state‑of‑the‑art cybersecurity agent development.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Oct 2

KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

KaliBench is a fine‑grained benchmark that evaluates large language models’ ability to translate natural‑language queries into executable command‑line instructions for Kali Linux. It contains 8,504 query–command pairs covering 1,642 tools across 23 capability dimensions and 5 security phases, and uses a deterministic, alias‑aware evaluation pipeline with multi‑stage verification (LLM validation, sandboxed execution, and human refinement). The benchmark also supports runtime‑free verifiable rewards, enabling training improvements that bring smaller models close to the performance of a 685B MoE model.

By Pengfei Li, Naufal Suryanto, Sicheng Zhang, Muzammal Naseer
arXiv AI
Jun 12

The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.

By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang