Autonomous cyber agents may be developed by applying reinforcement and deep reinforcement learning (RL/DRL), where agents are trained in a representative environment. The training environment must simulate with high-fidelity the network Cyber Operations (CyOp) that the agent aims to explore. Given the complexity of net-work CyOps, a good simulator is difficult to achieve. This work presents a systematic solution to automatically generate a high-fidelity simulator in the Cyber Gym for Intelligent Learning (CyGIL). Through representation learning and continuous learning, CyGIL provides a unified CyOp training environment where an emulated CyGIL-E automatically generates a simulated CyGIL-S. The simulator generation is integrated with the agent training process to further reduce the required agent training time. The agent trained in CyGIL-S is transferrable directly to CyGIL-E showing full transferability to the emulated "real" network. Experimental results are presented to demonstrate the CyGIL training performance. Enabling offline RL, the CyGIL solution presents a promising direction towards sim-to-real for leveraging RL agents in real-world cyber networks.
Cite
@article{arxiv.2304.01244,
title = {Unified Emulation-Simulation Training Environment for Autonomous Cyber Agents},
author = {Li Li and Jean-Pierre S. El Rami and Adrian Taylor and James Hailing Rao and Thomas Kunz},
journal= {arXiv preprint arXiv:2304.01244},
year = {2023}
}
Comments
To be published in the Proceedings of the 5th International Conference on Machine Learning for Networking (MLN'2022)