Estimating the number of clusters in multivariate data by self-organizing maps

Area of Science:

• Data Science
• Computer Vision
• Machine Learning

Background:

• Determining data structure without prior cluster knowledge is challenging across diverse fields like image analysis and biology.
• Traditional clustering methods face computational complexity with large datasets and unknown cluster numbers.
• Self-organizing feature maps (SOM) offer visualization but lack automated knowledge discovery.

Purpose of the Study:

• To propose a novel post-processing technique for U-matrix visualization from SOM unsupervised learning.
• To automate the identification and labeling of clusters within multivariate data.
• To enable classification of new data points based on identified cluster structures.

Main Methods:

• Utilizing region-based image processing techniques on the U-matrix generated by SOM.
• Applying mathematical morphology to identify similar neuron regions within the U-matrix.
• Developing an automated process to determine the number and labels of clusters.

Main Results:

• The proposed method successfully identifies distinct regions corresponding to data clusters.
• Automatic determination of cluster numbers and labels is achieved.
• New data points can be classified by mapping them to the best matching neuron.

Conclusions:

• The integration of image processing and mathematical morphology offers an effective solution for automated cluster discovery.
• This approach enhances the utility of SOM for knowledge discovery in multivariate data analysis.
• The method provides a robust framework for data partitioning and classification without prior assumptions.