This work aims to enable autonomous agents for network cyber operations (CyOps) by applying reinforcement and deep reinforcement learning (RL/DRL). The required RL training environment is particularly challenging, as it must balance the need for high-fidelity, best achieved through real network emulation, with the need for running large numbers of training episodes, best achieved using simulation. A unified training environment, namely the Cyber Gym for Intelligent Learning (CyGIL) is developed where an emulated CyGIL-E automatically generates a simulated CyGIL-S. From preliminary experimental results, CyGIL-S is capable to train agents in minutes compared with the days required in CyGIL-E. The agents trained in CyGIL-S are transferrable directly to CyGIL-E showing full decision proficiency in the emulated "real" network. Enabling offline RL, the CyGIL solution presents a promising direction towards sim-to-real for leveraging RL agents in real-world cyber networks.
@article{arxiv.2304.01366,
title = {Enabling A Network AI Gym for Autonomous Cyber Agents},
author = {Li Li and Jean-Pierre S. El Rami and Adrian Taylor and James Hailing Rao and Thomas Kunz},
journal= {arXiv preprint arXiv:2304.01366},
year = {2023}
}
Comments
To appear in Proceedings of the 2022 International Conference on Computational Science and Computational Intelligence