The paper investigates how frontier AI systems perform on tasks related to cyber security, specifically exploit generation, vulnerability repair, and subsequent attacks, using five nonpublic software environments. It evaluates both open‑weight and proprietary models, employing deterministic graders rather than LLM judges to score performance. Results show significant variation across systems and vulnerability types, with repair scores higher than attack scores in two environments and lower in three, and highlight that passing an initial security test does not guarantee long‑term defense, as 92 of 524 defender test intervals still experienced successful exploits after the first one was stopped.
arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.
arXiv:2606. 14295v1 Announce Type: cross Abstract: Frontier AI systems are increasingly capable of cybersecurity tasks, including codebase inspection, vulnerability detection, and exploitation.
arXiv:2607. 25379v1 Announce Type: new Abstract: Cyber-capable AI agents combine language models with tools, memory, and execution en- vironments to perform multi-step offensive-security tasks.
arXiv:2606. 04460v1 Announce Type: cross Abstract: AI has the potential to transform cybersecurity by enabling systems that can autonomously detect, analyze, and remediate software vulnerabilities.
arXiv:2606. 02644v1 Announce Type: cross Abstract: Agentic scaffolds have dramatically improved LLM performance on complex, long-horizon tasks, yielding both broad benefits and amplified risks in domains like cybersecurity.
Frontier AI model developers increasingly rely on layered safeguards to prevent catastrophic misuse, but little public evidence exists on how much protection these safeguards provide, or how consistently across developers. We introduce the FAR.
OpenAI is investing in stronger safeguards and defensive capabilities as AI models become more powerful in cybersecurity. We explain how we assess risk, limit misuse, and work with the security community to strengthen cyber resilience.
arXiv:2605. 10834v2 Announce Type: replace Abstract: AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will perform best in real-world targets.
arXiv:2608. 06471v1 Announce Type: cross Abstract: Despite recent advances, frontier large language model (LLM) agents remain limited in discovering and patching complex vulnerabilities in real-world software.
arXiv:2605. 11047v2 Announce Type: replace-cross Abstract: Agentic language-model systems increasingly rely on mutable execution contexts, including files, memory, tools, skills, and auxiliary artifacts, creating security risks beyond explicit user prompts.
The article discusses a notable imbalance in AI security research, where studies on attacking AI systems outnumber those on defending them. It highlights that this skew is evident across various subfields such as federated learning, speech recognition, membership inference, and large language models. The authors argue that attack papers often benefit from favorable evaluation conditions, whereas defense papers face stricter standards, resulting in a literature rich in vulnerabilities but lacking robust, deployable protections.
arXiv:2609.23894v1 Announce Type: cross Abstract: Agentic AI extends LLM security beyond generated content to persistent state, autonomous actions, tool use, and interactions with humans and other ag...