Application of Improved Asynchronous Advantage Actor Critic Reinforcement Learning Model on Anomaly Detection.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Cybersecurity

Background:

• Anomaly detection traditionally relies on mathematical and statistical methods.
• Reinforcement learning (RL) has shown significant success in various domains like gaming.
• Limited research exists on applying RL to anomaly detection.

Purpose of the Study:

• To propose an adaptable asynchronous advantage actor-critic (A3C) model for anomaly detection.
• To evaluate and compare the proposed RL model against classical machine learning and generative adversarial models.
• To differentiate sequence and image anomalies using specialized neural networks.

Main Methods:

• Introduction to basic principles of related anomaly detection models.
• Detailed problem definitions, modeling processes, and testing procedures for the proposed A3C model.
• Implementation of attention mechanisms and convolutional neural networks for sequence and image anomaly differentiation, respectively.

Main Results:

• The proposed A3C model demonstrated higher rewards and lower loss rates during training and testing.
• Performance metrics including precision, recall rate, and F1 score were superior or comparable to state-of-the-art models.
• Evaluation conducted on public benchmark datasets: NSL-KDD, AWID, CICIDS-2017, and DoHBrw-2020.

Conclusions:

• The proposed reinforcement learning model is effective for anomaly detection.
• The A3C model achieves comparable or superior results compared to existing anomaly detection techniques.
• The model's ability to handle diverse anomaly types (sequence, image) contributes to its effectiveness.