Silhouette width using generalized mean-A flexible method for assessing clustering efficiency

Area of Science:

• Data Mining
• Pattern Recognition
• Machine Learning

Background:

• Silhouette width is a key metric for cluster analysis, evaluating object fit and cluster quality.
• The standard silhouette method favors spherical clusters, limiting its application to complex, non-spherical data structures common in real-world scenarios.

Purpose of the Study:

• To generalize the silhouette width using the generalized mean to accommodate non-spherical clusters.
• To introduce a tunable parameter (p) that adjusts the index's sensitivity to cluster compactness and connectedness.

Main Methods:

• The generalized mean was implemented to modify the silhouette width calculation.
• The performance of the generalized silhouette width was evaluated on artificial and Iris datasets using various clustering algorithms and parameter (p) values.

Main Results:

• Negative values of 'p' allowed well-separated, non-spherical clusters to achieve high silhouette widths.
• Positive 'p' values amplified the preference for spherical clusters.
• The optimal clustering method varied with 'p', with single linkage favored at low 'p' and others at higher values.

Conclusions:

• The generalized silhouette width offers flexibility in evaluating cluster quality, particularly for complex data.
• This approach prevents the underestimation of clustering efficiency with heterogeneous or non-spherical clusters.
• The method allows for adjusting the balance between compactness and connectedness criteria in cluster assessment.