Cross-Clustering: A Partial Clustering Algorithm with Automatic Estimation of the Number of Clusters

Area of Science:

• Data Science
• Computational Biology
• Statistics

Background:

• Clustering methods often struggle with outlier detection, require pre-defined cluster numbers, and are sensitive to initialization.
• Existing algorithms lack robustness in identifying inappropriate data partitioning and can yield dependent results.

Purpose of the Study:

• To introduce Cross-clustering (CC), a novel partial clustering algorithm designed to overcome common limitations in data clustering.
• To enhance the accuracy of cluster identification, outlier detection, and membership determination.

Main Methods:

• CC combines principles from Ward's minimum variance and Complete-linkage hierarchical clustering.
• The algorithm was validated against existing methods using both simulated and real-world datasets.

Main Results:

• CC demonstrates superior performance in identifying the correct number of clusters and accurately detecting outliers.
• The method shows improved determination of true cluster memberships compared to traditional algorithms.
• CC proved effective in identifying disease subtypes and gene expression patterns.

Conclusions:

• Cross-clustering (CC) offers a robust and versatile solution for various data clustering challenges, including biological and non-biological applications.
• The CC algorithm is freely available in R, promoting wider adoption and research.