Cluster analysis: significance, empty space, clustering tendency, non-uniformity. II--Empty Space index

Area of Science:

• Data Science
• Statistical Analysis
• Information Theory

Background:

• Clustering indexes often conflate data separation with the presence of empty space.
• Existing methods struggle to accurately measure the degree of clustering versus data uniformity.
• Ambiguity in clustering evaluation arises from the undefined nature of 'empty space' within data sets.

Purpose of the Study:

• To introduce the Empty Space (ES) index for quantifying unoccupied regions in information space.
• To demonstrate ES index's utility in disambiguating traditional clustering metrics.
• To propose a corrected Minimum Spanning Tree (MST) index using ES for reliable clustering assessment.

Main Methods:

• The Empty Space (ES) index is defined as the fraction of information space distant from experimental points.
• ES quantifies regions where distances exceed the mean inter-point distance.
• The ES index is applied to correct the Minimum Spanning Tree (MST) index.

Main Results:

• The ES index effectively measures the proportion of unoccupied information space.
• ES disambiguates clustering indexes by isolating the impact of empty space.
• The corrected MST index, incorporating ES, provides a reliable measure of data clustering.

Conclusions:

• The Empty Space (ES) index offers a novel approach to understanding data distribution.
• ES enhances the interpretability and accuracy of clustering analysis.
• The corrected MST index presents a robust tool for evaluating data set clustering.