An improved X-means and isolation forest based methodology for network traffic anomaly detection

Area of Science:

• Computer Science
• Network Security
• Machine Learning

Background:

• Anomaly detection in large-scale networks is challenging due to complexity and diverse applications.
• Unsupervised learning algorithms are crucial for network anomaly detection, especially with unlabeled data.
• The accuracy of prior knowledge about abnormal ratios significantly impacts unsupervised algorithm performance.

Purpose of the Study:

• To propose a novel unsupervised anomaly detection algorithm, X-iForest, for network traffic.
• To enhance anomaly detection accuracy by leveraging X-means for secondary filtering.
• To evaluate the performance of X-iForest against mainstream unsupervised algorithms.

Main Methods:

• Developed the X-iForest algorithm combining X-means and Isolation Forest (iForest).
• Utilized X-means to cluster Euclidean distances between abnormal points and normal cluster centers for secondary filtering.
• Compared X-iForest with seven existing unsupervised algorithms.

Main Results:

• X-iForest demonstrated notable advantages over seven mainstream unsupervised algorithms.
• The algorithm achieved high accuracy in anomaly detection rates and AUC.
• Experimental results confirm X-iForest's effectiveness for large-scale network traffic data.

Conclusions:

• X-iForest is a robust and effective algorithm for unsupervised anomaly detection in network traffic.
• The proposed method offers significant improvements over existing techniques, particularly for large-scale datasets.
• X-iForest provides a valuable tool for enhancing network security and monitoring.