arXiv Computation and Language

CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild

arXiv AI

Jul 2

Toward Cybersecurity-Expert Small Language Models

arXiv:2510. 14113v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are transforming everyday applications, yet deployment in cybersecurity lags due to a lack of high-quality, domain-specific models and training datasets.

By Matan Levi, Daniel Ohayon, Ariel Blobstein, Ravid Sagi, Ian Molloy, Yair Allouche

llms benchmarks

arXiv AI

Aug 10

CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training

arXiv:2608. 06471v1 Announce Type: cross Abstract: Despite recent advances, frontier large language model (LLM) agents remain limited in discovering and patching complex vulnerabilities in real-world software.

By Amine Lbath, Manan Suri, Aurelien Delaitre, Vadim Okun, Massih-Reza Amini, Ram D. Sriram, Dinesh Manocha

llms agents fine-tuning

arXiv AI

Jul 1

A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems

arXiv:2606. 31639v1 Announce Type: cross Abstract: Large language models are no longer only text generators.

By Seyed Bagher Hashemi Natanzi, Bo Tang

llms agents robotics safety

arXiv AI

Jun 12

The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.

By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang

llms agents safety

arXiv AI

Jul 3

Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens

arXiv:2507. 02964v2 Announce Type: replace-cross Abstract: The increasing scale of AI workloads demands High-Performance Computing (HPC) infrastructure and training methodologies that are both scalable and sustainable.

By Salahuddin Salahuddin, Ahmed Hussain, Jussi L\"opp\"onen, Toni Jutila

llms fine-tuning benchmarks

arXiv AI

Jul 8

SecureCode: A Production-Grade Multi-Turn Dataset for Training Security-Aware Code Generation Models

arXiv:2512. 18542v3 Announce Type: replace-cross Abstract: AI coding assistants produce vulnerable code in 45\% of security-relevant scenarios~\cite{veracode2025}, yet no public training dataset teaches both traditional web security and AI/ML-specific defenses in a format suitable for instruction tuning.

By Scott Thornton

llms agents fine-tuning

arXiv AI

Oct 2

KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

KaliBench is a fine‑grained benchmark that evaluates large language models’ ability to translate natural‑language queries into executable command‑line instructions for Kali Linux. It contains 8,504 query–command pairs covering 1,642 tools across 23 capability dimensions and 5 security phases, and uses a deterministic, alias‑aware evaluation pipeline with multi‑stage verification (LLM validation, sandboxed execution, and human refinement). The benchmark also supports runtime‑free verifiable rewards, enabling training improvements that bring smaller models close to the performance of a 685B MoE model.

By Pengfei Li, Naufal Suryanto, Sicheng Zhang, Muzammal Naseer

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Sep 23

SelfOp: An Optimization Algorithm for Self-Improving Security Agents

arXiv:2609.22792v1 Announce Type: cross Abstract: LLM agents are increasingly used for security tasks: vulnerability discovery, exploit reproduction, and patch generation. Improving them at the model...

By Saad Ullah, Yigitcan Kaya, Christopher Kruegel, Giovanni Vigna, Gianluca Stringhini

llms agents benchmarks

arXiv AI

Sep 15

Data Security in Large Language Models: Risks, Defense, and Directions

arXiv:2508.02312v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs), now a foundation in advancing natural language processing, power applications such as text generation, machine...

By Kang Chen, Xiuze Zhou, Yuanhui Yu, Yuanguo Lin, Hefeng Chen, Congyu Cai, Li Shen

llms rag agents nlp reinforcement-learning safety

arXiv AI

Jul 23

Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis

arXiv:2607. 20216v1 Announce Type: cross Abstract: Malware analysis demands rapid interpretation of complex detonation reports spanning filesystem, network, and process behaviours.

By Adel ElZemity, Shujun Li, Budi Arief

llms agents benchmarks safety

arXiv AI

Jul 9

Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategies

arXiv:2607. 06963v1 Announce Type: cross Abstract: Large Language Models (LLMs) and generative AI (GenAI) systems, such as ChatGPT, Claude, Gemini, LLaMA, Copilot, Stable Diffusion by OpenAI, Anthropic, Google, Meta, Microsoft, Stability AI, respectively, are revolutionizing cybersecurity, enabling both automated defense and sophisticated attacks.

By Kiarash Ahi, Saeed Valizadeh

llms diffusion benchmarks safety

arXiv AI

Jun 3

Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward

arXiv:2602. 12430v4 Announce Type: replace-cross Abstract: The transition from monolithic language models to modular, skill-equipped agents marks a defining shift in how large language models (LLMs) are deployed in practice.

By Renjun Xu, Yang Yan

llms agents reinforcement-learning benchmarks

Related stories