Ranking-enhanced anomaly detection using Active Learning-assisted Attention Adversarial Dual AutoEncoder.

Area of Science:

• Cybersecurity
• Machine Learning
• Anomaly Detection

Background:

• Advanced Persistent Threats (APTs) present persistent cybersecurity challenges due to their stealthy nature.
• Supervised learning for APT detection requires substantial labeled data, which is often unavailable in real-world scenarios.
• Existing methods struggle with highly imbalanced datasets common in cybersecurity.

Purpose of the Study:

• To develop an effective unsupervised anomaly detection framework for identifying APTs.
• To leverage active learning to minimize the need for extensive manual data labeling.
• To enhance the accuracy and efficiency of APT detection in complex environments.

Main Methods:

• Utilized AutoEncoders for unsupervised anomaly detection.
• Integrated an active learning loop to iteratively refine the detection model by querying uncertain samples.
• Proposed an Attention Adversarial Dual AutoEncoder framework for anomaly detection.
• Evaluated the framework on imbalanced real-world provenance trace datasets.

Main Results:

• Achieved significant improvements in detection rates through active learning.
• Demonstrated superior performance compared to existing anomaly detection approaches.
• Successfully detected APT-like attacks in highly imbalanced datasets (as low as 0.004% of data).
• Validated the framework across multiple operating systems (Android, Linux, BSD, Windows).

Conclusions:

• The proposed AutoEncoder and active learning approach effectively detects APT anomalies with reduced labeling costs.
• The framework offers a promising solution for cybersecurity anomaly detection in data-scarce and imbalanced environments.
• Active learning iteratively enhances model performance, making it adaptable to evolving threats.