Clustering high-dimensional data via feature selection.

Area of Science:

• Statistics
• Machine Learning
• Bioinformatics

Background:

• High-dimensional data analysis, including microarray and RNA-seq data, presents significant clustering challenges.
• Existing methods may struggle with identifying relevant features in large datasets.

Purpose of the Study:

• To propose and evaluate a novel clustering procedure, spectral clustering with feature selection (SC-FS), for high-dimensional data.
• To demonstrate the method's ability to identify informative features and achieve optimal clustering error rates.

Main Methods:

• Spectral clustering is initially used to estimate data labels.
• Features with the highest R-squared values relative to these labels are selected.
• A second clustering round is performed using only the selected features.

Main Results:

• The SC-FS method is theoretically proven to identify informative features with high probability under mild conditions.
• The procedure achieves a minimax optimal clustering error rate for the sparse Gaussian mixture model.
• Empirical validation on four real-world datasets confirms the method's effectiveness for high-dimensional data.

Conclusions:

• SC-FS offers a robust and effective approach for clustering high-dimensional data.
• The feature selection component enhances clustering performance and interpretability.
• This method has broad applicability in fields utilizing large-scale biological data.