Selective ensemble method for anomaly detection based on parallel learning.

Area of Science:

• Data Science
• Machine Learning
• Artificial Intelligence

Background:

• Anomaly detection is crucial in data analysis, but traditional methods struggle with data characteristics.
• Ensemble methods enhance generalization but face challenges like data imbalance and resource demands.
• Existing ensemble anomaly detection techniques require careful selection of base detectors.

Purpose of the Study:

• To propose a novel selective ensemble method for anomaly detection based on parallel learning (SEAD-PL).
• To address key challenges in ensemble anomaly detection, including data imbalance, time/space complexity, and base detector selection.
• To improve the generalization ability and performance of anomaly detection systems.

Main Methods:

• Implemented a differentiated stratified sampling method to mitigate data imbalance.
• Developed a distributed parallel training framework to optimize time and space efficiency for base detector training.
• Utilized a clustering-based ensemble selection strategy to balance detector accuracy and diversity.

Main Results:

• The proposed SEAD-PL method demonstrated significant advantages over four other selected methods.
• Experiments conducted on six diverse datasets validated the effectiveness of the SEAD-PL approach.
• The method successfully alleviated data imbalance and reduced computational resource requirements.

Conclusions:

• The SEAD-PL method offers an effective solution for anomaly detection, particularly in scenarios with imbalanced data and large datasets.
• The parallel learning framework and clustering-based selection strategy enhance the performance and scalability of ensemble anomaly detection.
• This approach provides a robust and efficient alternative to traditional anomaly detection techniques.