Simultaneous clustering and variable selection: A novel algorithm and model selection procedure.

Area of Science:

• Behavioral Science
• Data Mining
• Bioinformatics

Background:

• High-dimensional data offer new research opportunities but present challenges with irrelevant variables.
• Irrelevant variables can obscure cluster structures and hinder accurate subgroup discovery.
• Existing clustering methods struggle with simultaneous variable selection and robust partitioning.

Purpose of the Study:

• Introduce a novel clustering algorithm, Cardinality K-means (CKM), for high-dimensional data.
• Develop a new model selection strategy for accurate cluster number determination.
• Address the challenge of irrelevant variables in clustering analysis.

Main Methods:

• Developed the Cardinality K-means (CKM) algorithm for simultaneous clustering and variable selection.
• Proposed a novel model selection strategy focusing on signaling variables to determine the number of clusters.
• Validated CKM and the selection strategy through simulation studies and genetic data analysis.

Main Results:

• CKM demonstrated high stability in performing clustering and variable selection.
• CKM consistently outperformed competing methods in recovering cluster partitions and identifying signaling variables.
• The novel model selection strategy provided more accurate cluster number estimation compared to conventional methods.

Conclusions:

• CKM offers a robust solution for clustering high-dimensional data by integrating variable selection.
• The proposed model selection strategy enhances the accuracy of determining the number of clusters.
• The CKM algorithm and model selection strategy are available in an R package for broader accessibility.