arXiv:2508.18370v3 Announce Type: replace-cross
Abstract: Large language models (LLMs) have demonstrated exceptional capabilities when trained within executable runtime environments, notably excellin...
By Terry Yue Zhuo, Dingmin Wang, Hantian Ding, Varun Kumar, Zijian Wang
arXiv:2608.23181v1 Announce Type: cross
Abstract: As large language models (LLMs) continue to advance in coding capabilities, their potential in cybersecurity has drawn increasing research attention,...
By Jian Yang, Haau-Sing Li, Shawn Guo, Zixi Zhao, Yibo Tan, Jiajun Wu, Aishan Liu, Xianglong Liu, Tianyu Zheng, Bryan Dai, Chengran Yang
arXiv:2610.03153v1 Announce Type: cross
Abstract: Workspace agents combine large language models with execution harnesses to perform stateful, multi-step tasks that access or modify external resource...
By Shiyi Kuang, Xuemei Luo, Kun Liu, Junhai Li, Rui Tian, Feng Shi, Bo Shen, Nianyu Li, Dehui Li, Ping Chen
arXiv:2605. 26195v2 Announce Type: replace-cross Abstract: LLM-based agents are increasingly used for cybersecurity tasks, but most existing systems rely on fixed, human-designed scaffolds that struggle to adapt across diverse targets and failure modes.
By Yihe Fan, Changyi Li, Lichen Xu, Xudong Pan, Jiarun Dai, Hong Geng, Min Yang
KaliBench is a fine‑grained benchmark that evaluates large language models’ ability to translate natural‑language queries into executable command‑line instructions for Kali Linux. It contains 8,504 query–command pairs covering 1,642 tools across 23 capability dimensions and 5 security phases, and uses a deterministic, alias‑aware evaluation pipeline with multi‑stage verification (LLM validation, sandboxed execution, and human refinement). The benchmark also supports runtime‑free verifiable rewards, enabling training improvements that bring smaller models close to the performance of a 685B MoE model.
By Pengfei Li, Naufal Suryanto, Sicheng Zhang, Muzammal Naseer
arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.
By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang