Variance-Based Cluster Selection Criteria in a K-Means Framework for One-Mode Dissimilarity Data.

Area of Science:

• Statistics
• Data Mining
• Machine Learning

Background:

• Determining the optimal number of clusters is a fundamental challenge in cluster analysis.
• Existing methods, like those for K-means, often rely on two-mode data sets and total point scatter decomposition.
• Adapting these criteria for one-mode dissimilarity matrices, where object coordinates are unknown, requires reformulation.

Purpose of the Study:

• To formulate novel criteria for determining the number of clusters using one-mode dissimilarity matrices.
• To adapt existing variance-based criteria for situations where only dissimilarities are available.
• To evaluate the performance of these new criteria compared to traditional methods.

Main Methods:

• Proposed decomposition of object variability based on block-shaped partitions of the dissimilarity matrix.
• Derived within-block and between-block dispersion values from the partitioned dissimilarity matrix.
• Formulated new variance-based criteria for cluster number determination.
• Conducted Monte Carlo simulations to assess criterion performance.

Main Results:

• The proposed variance-based criteria, when applied to Euclidean distances derived from one-mode data, showed greater efficiency in recovering the number of clusters.
• This improved efficiency was particularly notable for unequal-sized clusters and in low-dimensional settings.
• When applied directly to dissimilarity data, the proposed criteria consistently outperformed their original formulations.

Conclusions:

• The developed variance-based criteria offer an effective approach for determining the number of clusters from one-mode dissimilarity matrices.
• Reformulating criteria using derived Euclidean distances can enhance cluster recovery accuracy, especially in challenging data configurations.
• These findings provide valuable tools for researchers dealing with dissimilarity data in cluster analysis.