Machine-learned cluster identification in high-dimensional data.

Area of Science:

• Bioinformatics
• Computational Biology
• Data Science

Background:

• High-dimensional biomedical data analysis often involves clustering to identify disease subtypes.
• Classical clustering algorithms may impose artificial structures or misclassify data points due to predefined shape assumptions.
• Emergent self-organizing feature maps (ESOM) offer a potential alternative to address these limitations.

Purpose of the Study:

• To evaluate if Emergent Self-Organizing Feature Maps (ESOM) can avoid erroneous cluster identification in biomedical data.
• To compare the performance of ESOM/U-matrix with classical clustering algorithms on various data complexities.

Main Methods:

• ESOM analysis was performed on diverse datasets using an interactive R-based bioinformatics tool.
• The U-matrix was employed to visualize distance structures in the high-dimensional feature space.
• Clustering results were compared against single linkage, Ward, and k-means algorithms.

Main Results:

• Ward clustering incorrectly identified structures in random (golf ball, cuboid, S-shaped) and permuted real-world data.
• ESOM/U-matrix accurately identified the absence of structure in random data and correctly detected true clusters in biomedical datasets.
• Classical algorithms demonstrated a tendency for erroneous results on simple and complex data, indicating potential failures in high-dimensional biomedical data analysis.

Conclusions:

• Classical hierarchical clustering algorithms exhibit a significant propensity for generating erroneous results.
• Unsupervised machine learning via the ESOM/U-matrix method provides a robust and unbiased approach for identifying true clusters in high-dimensional complex data.