Hierarchical Sparse Representation Clustering for High-Dimensional Data Streams

Area of Science:

• Data Science
• Machine Learning
• Big Data Analytics

Background:

• Data stream clustering is crucial for identifying patterns in continuous data.
• Existing algorithms struggle with high-dimensional data streams due to distance metric limitations and noise sensitivity.
• Intractability in similarity measurement and noise sensitivity pose significant challenges for current high-dimensional data stream clustering methods.

Purpose of the Study:

• To propose a novel Hierarchical Sparse Representation Clustering (HSRC) framework for high-dimensional data streams.
• To address the challenges of Euclidean distance limitations and noise sensitivity in existing algorithms.
• To enable effective clustering and outlier detection in complex, high-dimensional data streams.

Main Methods:

• Employs a sparse representation-based technique to learn an affinity matrix within landmark windows.
• Utilizes spectral clustering on the affinity matrix to form initial microclusters.
• Merges microclusters into macroclusters using sparse similarity degrees (SSDs) and refines them via fine-tuning, incorporating sparsity residual values (SRVs) for outlier detection and representative selection.

Main Results:

• The HSRC framework demonstrates effectiveness in clustering high-dimensional data streams.
• Sparsity residual values (SRVs) enable adaptive selection of representative data objects and robust outlier detection.
• Experimental results on benchmark datasets confirm the framework's robustness and superior performance compared to existing methods.

Conclusions:

• The proposed Hierarchical Sparse Representation Clustering (HSRC) framework effectively addresses the challenges of high-dimensional data stream clustering.
• HSRC provides a robust approach for pattern discovery and outlier detection in continuous, high-dimensional data.
• The framework's innovative use of sparse representation and spectral clustering offers a significant advancement in the field.