Infrequent Pattern Detection for Reliable Network Traffic Analysis Using Robust Evolutionary Computation.

Area of Science:

• Cybersecurity
• Machine Learning
• Data Mining

Background:

• Anomaly detection is crucial in cybersecurity but challenged by rare, computationally expensive infrequent patterns.
• Cybersecurity datasets often contain numerous irrelevant features, hindering machine learning performance.
• Feature selection (FS) is vital for preprocessing cybersecurity data to improve analysis.

Purpose of the Study:

• To apply a cooperative co-evolution-based feature selection with random feature grouping (CCFSRFG) to a cybersecurity dataset.
• To evaluate the impact of FS on unsupervised infrequent pattern detection (UIPD) using 10 anomaly detection techniques.
• To compare the true positive rate (TPR) of UIPD with and without FS.

Main Methods:

• Applied CCFSRFG for feature selection on a benchmark cybersecurity dataset.
• Utilized 10 unsupervised anomaly detection techniques for infrequent pattern detection.
• Compared performance metrics, primarily TPR, between datasets with and without FS.

Main Results:

• Feature selection led to substantial improvements in TPR for detecting infrequent patterns.
• The cluster-based local outlier factor (CBLOF) achieved a 385.91% TPR improvement for backdoor pattern detection with FS.
• The clustering-based multivariate Gaussian outlier score (CMGOS) demonstrated the highest overall TPR improvement of 61.47% with FS.

Conclusions:

• Cooperative co-evolution-based feature selection is an effective preprocessing step for cybersecurity data.
• FS significantly enhances the performance of unsupervised anomaly detection techniques for identifying infrequent patterns.
• The proposed Unsupervised Infrequent Pattern Detection (UIPD) framework, augmented with FS, offers a more efficient and accurate solution for cybersecurity anomaly detection.