Learning Self-Growth Maps for Fast and Accurate Imbalanced Streaming Data Clustering.

Area of Science:

• Data Mining
• Machine Learning
• Artificial Intelligence

Background:

• Streaming data analysis presents challenges due to dynamic cluster imbalance, where cluster imbalance ratios (IR) change over time.
• This imbalance can negatively impact the accuracy and efficiency of traditional streaming data clustering algorithms.
• Existing methods struggle to adapt to evolving data distributions and maintain performance.

Purpose of the Study:

• To develop an accurate and efficient streaming data clustering approach that adapts to dynamic and imbalanced cluster distributions.
• To address the limitations of current methods in handling evolving data and maintaining performance.
• To propose a novel algorithm capable of incremental adaptation and accurate identification of imbalanced clusters.

Main Methods:

• A self-growing map (SGM) was designed to dynamically arrange neurons based on local data distribution, enabling fast, incremental adaptation.
• SGM utilizes density-sensitive neurons to capture global distributions and prevent the omission of small clusters in imbalanced datasets.
• A fast hierarchical merging (HM) strategy was developed, leveraging the SGM for efficient retrieval of intracluster distribution pairs, avoiding computationally expensive global searches.

Main Results:

• The proposed SGM demonstrates incremental adaptation capabilities to new data chunks in streaming environments.
• The self-growth map-guided hierarchical merging for imbalanced data clustering (SOHI) approach efficiently determines the true number of imbalanced clusters.
• Extensive experiments confirm that SOHI achieves both high efficiency and accuracy in exploring cluster distributions for streaming data.

Conclusions:

• The SGM effectively manages evolving data distributions in real-time streaming scenarios.
• SOHI provides a robust solution for accurately and efficiently clustering imbalanced streaming data.
• The developed approach significantly advances the field of data mining for dynamic and imbalanced datasets.