Dynamic Sparse Subspace Clustering for Evolving High-Dimensional Data Streams.

Area of Science:

• Data Science
• Machine Learning
• Artificial Intelligence

Background:

• Large-scale data streams present challenges for traditional analysis methods.
• High-dimensional data often reside in unions of low-dimensional subspaces.
• Existing data stream clustering methods struggle with dynamic subspace changes.

Purpose of the Study:

• To propose a novel sparse representation-based data stream clustering algorithm.
• To address the time-varying nature of subspaces in evolving data streams.
• To improve both accuracy and efficiency in clustering large data streams.

Main Methods:

• Developed evolutionary dynamic sparse subspace clustering (EDSSC) with static learning and online clustering phases.
• Introduced EDSSC summary for efficient data stream statistics.
• Proposed an algorithm to estimate the number of subspaces, eliminating the need for prior knowledge.
• Utilized the average sparsity concentration index (ASCI) for enhanced clustering accuracy.
• Implemented a subspace evolution detection model using the Page-Hinkley test.

Main Results:

• EDSSC effectively handles subspace emergence, disappearance, and recurrence.
• The EDSSC summary balances data retention for accuracy and data reduction for efficiency.
• ASCI significantly improves clustering accuracy over traditional SCI.
• The algorithm successfully detects and adapts to evolving subspaces.
• Experimental results demonstrate superior performance compared to state-of-the-art online subspace clustering approaches.

Conclusions:

• EDSSC offers a robust solution for clustering evolving high-dimensional data streams.
• The method is adaptive to dynamic changes in data subspaces.
• EDSSC achieves high accuracy and efficiency, outperforming existing methods on real-world data.