Case count metric for comparative analysis of entity resolution results

Area of Science:

• Data Science
• Computer Science

Background:

• Entity resolution (ER) systems often yield varying clustering results due to changes in parameters, algorithms, or configurations.
• Evaluating ER accuracy is challenging in real-world scenarios where the true data linking structure is unknown.

Purpose of the Study:

• To introduce the Case Count Metric System (CCMS) for comparing two ER clustering outcomes.
• To provide a method for evaluating ER systems without relying on a ground truth dataset.

Main Methods:

• CCMS is a process and software system designed to compare ER clustering results.
• It classifies cluster transformations into four categories: unchanged, merged, partitioned, or overlapping.
• The system generates aggregate counts, singleton summaries, and per-cluster details for analysis.

Main Results:

• CCMS was applied to synthetic demographic and industrial materials datasets.
• The system successfully identified changes in clustering outcomes resulting from parameter adjustments and alternative ER systems.
• CCMS offers detailed insights into cluster reorganizations, including over-linking and under-linking.

Conclusions:

• CCMS offers a practical and interpretable approach for comparing ER clustering results when ground truth data is unavailable.
• It provides more actionable insights than traditional single-value similarity measures.
• CCMS supports both research analysis and operational ER system evaluation.