Adaptive windowing based recurrent neural network for drift adaption in non-stationary environment.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Data streams present challenges due to their massive volume and immediate processing needs.
• Concept drift, where data distribution changes over time, degrades the performance of traditional classification models.
• Neural networks are suitable for processing large, non-stationary data streams.

Purpose of the Study:

• To develop an adaptive windowing approach for recurrent neural networks to address concept drift in data streams.
• To improve the performance and adaptability of models dealing with non-stationary data.
• To prevent catastrophic forgetting in incrementally trained models.

Main Methods:

• Proposed an adaptive windowing approach using a gated recurrent unit (GRU), a type of recurrent neural network.
• Implemented incremental training on incoming data for real-world airline and synthetic Streaming Ensemble Algorithm (SEA) datasets.
• Utilized elastic weight consolidation with the Fisher information matrix to mitigate forgetting.

Main Results:

• The proposed model dynamically adjusts window size: increasing it for correct predictions and decreasing it upon detecting drifts.
• Achieved superior accuracy compared to state-of-the-art methods on both the airline (67.74%) and SEA (91.70%) datasets.
• Demonstrated improved accuracy by 3.6% for SEA and 1.6% for the airline dataset.

Conclusions:

• The adaptive recurrent neural network model effectively handles concept drift in non-stationary data streams.
• The dynamic windowing strategy ensures consistent performance by adapting to data evolution.
• The proposed method offers a robust solution for real-time data stream classification challenges.