CybORG: Improving an environment for effective training of cybersecurity agents

The paper examines the development and testing of a training scenario for automated cyber defense (ACD) using reinforcement learning (RL). The authors focus on the application of the Proximal Policy Optimization (PPO) algorithm for agent training in the CybORG environment, analyzing the effectiveness of the proposed approach, and implementing fixes for critical flaws in the CybORG training environment. The paper presents the results of testing the scenario, identifies its weaknesses, and proposes improvements aimed at optimizing the training process. The paper demonstrates the results of the changes made which had a significant impact on the effectiveness of the CybORG training environment. The changes made to the CybORG training environment improved its performance and usability. The analysis demonstrates that the proposed modifications facilitate more efficient agent training and simplify the integration of new scenarios. Based on the results obtained, recommendations for further improvement of autonomous cyber operations training environments (ACOGyms) are formulated.