Classes are not Clusters: Improving Label-based Evaluation of Dimensionality Reduction.

Area of Science:

• Machine Learning
• Data Science
• Computational Statistics

Background:

• Evaluating dimensionality reduction (DR) embeddings often relies on class labels, assuming distinct clusters in original data.
• This assumption is frequently violated, with classes fragmenting or merging, compromising label-based DR evaluation reliability.
• Existing methods like Trustworthiness and Continuity may not accurately reflect DR performance when class structures are complex.

Purpose of the Study:

• Introduce novel quality measures, Label-Trustworthiness and Label-Continuity (Label-T&C), for DR embedding evaluation.
• Develop a method that assesses DR reliability by comparing cluster structures in original and embedded spaces.
• Provide a more robust and accurate assessment of DR embedding quality, especially when class assumptions are violated.

Main Methods:

• Label-T&C quantify cluster formation in both high-dimensional and embedded spaces.
• The core methodology involves estimating and comparing the degree of class clustering across different dimensionalities.
• These measures evaluate the consistency of cluster structures preserved by DR techniques.

Main Results:

• Label-T&C demonstrate superior accuracy compared to established DR evaluation metrics (e.g., Trustworthiness, Continuity, KL divergence).
• The proposed measures effectively assess how well DR embeddings preserve underlying cluster structures.
• Label-T&C exhibit scalability, making them suitable for large datasets.

Conclusions:

• Label-T&C offer a more reliable approach to evaluating DR embeddings, particularly when initial class separability assumptions do not hold.
• These novel metrics can reveal intrinsic properties of DR techniques and the impact of hyperparameter choices.
• The findings suggest Label-T&C as a valuable tool for advancing DR method development and application.