Mixture Model Tests Of Hierarchical Clustering Algorithms: The Problem Of Classifying Everybody.

Area of Science:

• Computational statistics
• Machine learning
• Data mining

Background:

• Outliers can negatively impact mixture model tests in hierarchical clustering.
• Traditional accuracy assessments may not fully capture algorithm performance.
• Need for robust comparison methods in hierarchical clustering analysis.

Purpose of the Study:

• To develop a more valid accuracy assessment for hierarchical clustering algorithms.
• To compare the performance of different clustering algorithms in resolving multivariate normal mixtures.
• To identify algorithms that are robust to outliers and provide accurate classifications.

Main Methods:

• Accuracy was calculated at multiple levels within the hierarchical tree.
• Accuracy was evaluated as a function of classification coverage.
• Ten multivariate normal mixtures were used to test algorithm performance.

Main Results:

• All tested algorithms significantly outperformed random linkage.
• Accuracy showed an inverse relationship with classification coverage.
• Correlation-based similarity measures yielded significantly higher accuracy than Euclidean distance (p < .001).

Conclusions:

• A coverage-dependent accuracy assessment provides more valid comparisons of hierarchical clustering algorithms.
• Correlation-based similarity measures are superior to Euclidean distance for resolving multivariate normal mixtures.
• Algorithms like single, average, centroid linkage (correlation), and Ward's minimum variance are highly accurate.