The paper investigates whether large language models (LLMs) can perform structured security reasoning in cybersecurity decision-making. By testing LLMs on defense selection over attack graphs from real-world threat scenarios, the study finds that LLMs can produce coherent strategies when the attack-graph structure is explicitly provided, yet their performance is fragile, highly sensitive to prompt framing, and deteriorates with increasing graph complexity. Additionally, LLM-generated solvers recover the correct high-level formulation but scale poorly compared to specialized solvers.
By Pasquale Malacaria, Yunxiao Zhang
arXiv:2607. 19742v1 Announce Type: cross Abstract: Cyber Threat Intelligence (CTI) reports richly describe real-world attack processes, but their unstructured narratives cannot be directly used for automated attack-path reasoning.
By Wenbo Hou, Ning Hu, Xueping Wang, Jiahao Gu, Wenjian Luo
arXiv:2608.23181v1 Announce Type: cross
Abstract: As large language models (LLMs) continue to advance in coding capabilities, their potential in cybersecurity has drawn increasing research attention,...
By Jian Yang, Haau-Sing Li, Shawn Guo, Zixi Zhao, Yibo Tan, Jiajun Wu, Aishan Liu, Xianglong Liu, Tianyu Zheng, Bryan Dai, Chengran Yang
arXiv:2609.15523v1 Announce Type: cross
Abstract: Logic attack graphs grounded in scanner output provide explicit and auditable attack path reasoning LLM-based agents lack. Integrating symbolic frame...
By Oliver Stevanovic, Jasmin Wachter
The paper introduces CTF-ABACUS, a trace-based auditing framework that reconstructs each autonomous language-model agent’s run in Capture-the-Flag (CTF) challenges into evidence‑grounded solve profiles. By decomposing actions into penetration‑testing phases and techniques, it distinguishes genuine exploitation from shortcut methods such as memorized recall or guessing. Applying the framework to 1,435 CTF attempts by six models on 240 challenges shows that only 62‑87% of recovered flags are trace‑verified, highlighting that many successes rely on shallow trajectories rather than true exploitation.
By Kimberly Milner, Minghao Shao, Nanda Rani, Haoran Xi, Venkata Sai Charan Putrevu, Meet Udeshi, Sandeep K. Shukla, Prashanth Krishnamurthy, Farshad Khorrami, Muhammad Shafique, Ramesh Karri
arXiv:2606. 14517v1 Announce Type: cross Abstract: LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents.
By Yuguang Zhou, Xunguang Wang, Pingchuan Ma, Zhantong Xue, Zhaoyu Wang, Shuai Wang
arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.
By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang
arXiv:2607. 07474v1 Announce Type: cross Abstract: Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not.
By Harry Owiredu-Ashley
Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists. We evaluate five defense paradigms (no defense, static steering, CAST, AlphaSteer, probe-gated) across seven instruction-tuned models (7-31B) and five attack types (GCG, AutoDAN, DeepInception, prefilling, intent laundering).
arXiv:2606. 29441v1 Announce Type: cross Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists.
By Subhadip Mitra
arXiv:2608. 10530v1 Announce Type: cross Abstract: Large Language Models (LLMs) have undergone a shift from stateless conversational interfaces to autonomous agents capable of multi-step planning, tool invocation, code execution, and maintaining persistent memory.
By Md Jafrin Hossain, Mohammad Arif Hossain, Nirwan Ansari
arXiv:2605. 17480v3 Announce Type: replace Abstract: Multi-agent systems extend large language models (LLMs) by decomposing tasks among specialized agents, but their distributed decision process creates new attack surfaces.
By Qiqi Liu, Runhan Song, Shilin Ye