An ensemble learning method with GAN-based sampling and consistency check for anomaly detection of imbalanced data...

Area of Science:

• Computer Science
• Machine Learning
• Data Mining

Background:

• Real-world data streams often exhibit imbalance and concept drift, posing significant challenges for anomaly detection.
• Existing anomaly detection methods frequently assume balanced data, neglecting the joint issue of imbalance and concept drift.
• Addressing nonstationary data streams requires robust techniques that can adapt to evolving data distributions.

Purpose of the Study:

• To propose a novel ensemble learning method, EGSCC, to effectively handle anomaly detection in imbalanced data streams with concept drift.
• To develop a comprehensive framework integrating generative adversarial network-based sampling and a consistency check module.
• To improve the accuracy and robustness of anomaly detection in dynamic and imbalanced environments.

Main Methods:

• Developed an anomaly detection framework featuring a generative adversarial network (GAN) for oversampling imbalanced data, incorporating double encoders.
• Implemented a stacking ensemble learning approach with base classifiers (SVM, KNN, DT, RF) and a meta-classifier (LR) to manage concept drift.
• Introduced a statistical consistency check for incremental instances and validation sets, dynamically updating the validation set.

Main Results:

• The proposed EGSCC method demonstrated significant advantages in anomaly detection performance on imbalanced data streams with concept drift.
• Evaluated on artificial and real-world datasets, the method showed superior detection performance, parameter sensitivity, algorithm cost, and anti-noise ability.
• Experimental results confirmed the effectiveness of the GAN-based sampling and consistency check in handling data stream challenges.

Conclusions:

• The EGSCC method provides an effective solution for anomaly detection in challenging imbalanced data streams experiencing concept drift.
• The integration of GAN-based oversampling and statistical consistency checking enhances the adaptability and accuracy of anomaly detection.
• This research contributes a robust framework for analyzing nonstationary and imbalanced data streams in various real-world applications.