The paper introduces Ask the Expert, a training-time framework that leverages large language models (LLMs) to guide reinforcement learning for autonomous cyber defense. It summarizes complex cyber‑defense states, intermittently queries an LLM for host‑level defensive recommendations via a constrained action interface, and then uses those recommendations to shape rewards for PPO training. After training, the LLM is discarded, leaving a pure RL policy that improves sample efficiency and outperforms potential‑based reward shaping baselines on TTCP CAGE CC1 and CC2 scenarios.
arXiv:2607. 28826v1 Announce Type: new Abstract: Autonomous Cyber Operations (ACO) are increasingly important for defending enterprise networks as cyber threats continue to evolve in sophistication.
Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost exclusively against static, heuristic red agents, leaving their robustness against adaptive threats critically understudied. Meanwhile, recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have improved LLM reasoning, but their integration into cybersecurity remains elusive due to the absence of suitable benchmark environments and interaction datasets.
The paper explores using frozen large language models (LLMs) to control hierarchical cyber defense systems without retraining across different network scales. By separating planning (selecting subnets to defend) from execution (choosing defensive actions), the authors compare RL+RL, LLM+RL, and LLM+LLM setups in the Cyberwheel environment. Results show that while an LLM planner alone offers limited gains, extending LLM control to execution—especially with a 70B general‑purpose model—yields strong defensive performance across small, medium, and large networks, matching or surpassing retrained RL baselines.
arXiv:2602. 04809v3 Announce Type: replace Abstract: Recent years have seen an explosion of interest in autonomous cyber defence agents trained to defend computer networks using deep reinforcement learning.
arXiv:2608. 04317v1 Announce Type: cross Abstract: Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost exclusively against static, heuristic red agents, leaving their robustness against adaptive threats critically understudied.
arXiv:2606. 18223v1 Announce Type: cross Abstract: With sophisticated cyber-attacks becoming increasingly prevalent, modern networks require intelligent autonomous cyber-defense agents trained via Reinforcement Learning (RL).
arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.
arXiv:2509. 20008v2 Announce Type: replace Abstract: Penetration testing, the simulation of cyberattacks to identify security vulnerabilities, presents a sequential decision-making problem well-suited for reinforcement learning (RL) automation.
arXiv:2606. 01991v1 Announce Type: new Abstract: As Large Language Model (LLM) agents increasingly leverage the Model Context Protocol (MCP) to operate in complex environments, the expansion of their action spaces offers agents unsafe capabilities and underscores the risk of power-seeking.
The study evaluates autonomous agents that respond to network intrusions within a cyber range designed for human operator training. Using an emulated network with variable topology, red‑team attacks, and simulated users, the agents aim to block unauthorized access while minimizing defensive costs. Experiments compare heuristic policies with reinforcement‑learning‑derived policies, finding that the latter generally defend more efficiently, though performance varies with adversary strategy and user simulation.
The paper introduces CoER, a framework that defends language‑model agents against adaptive indirect prompt injection (IPI) by employing attacker‑defender co‑evolution and refinement. CoER models IPI as a general‑sum Markov game, uses Co‑PPO to maintain historical opponent populations, and fine‑tunes defenders only on verified safe demonstrations. In experiments across seven domains, CoER cuts attack success from 38.5% to 0.2% while boosting task utility from 63.2% to 76.3%.
arXiv:2610.28989v1 Announce Type: cross Abstract: Customized agent deployments define policies over tools, resources, and execution conditions. Providing these policies in the task agent's prompt doe...