DCC: A Model-Free Frame to Evaluate Data Set Quality.

Area of Science:

• Data Science
• Materials Science
• Statistical Modeling

Background:

• Evaluating data set quality is crucial for reliable analysis and model performance.
• Existing methods for data quality assessment are often computationally intensive and model-dependent.
• There is a need for a theoretically grounded and widely applicable framework to evaluate data completeness and representativeness.

Purpose of the Study:

• To propose the Data Correlation Convergence (DCC) framework for evaluating data set quality.
• To offer an alternative to conventional computation-intensive and model-dependent approaches.
• To quantify data set stability under perturbations, reflecting completeness and representativeness.

Main Methods:

• DCC integrates multiple correlation functions to quantify numeric correlations and distributional similarities.
• The framework hypothesizes that high-quality data sets exhibit stable correlation patterns under perturbations.
• Hypothetical and benchmark data sets were used to validate the DCC framework's efficacy.

Main Results:

• The lowest DCC values were observed at 10-20% linear correlations, increasing with more determinative correlations.
• DCC values effectively predict performance metrics (e.g., accuracy, R-squared) and feature importance (SHAP values) for machine learning models.
• DCC can efficiently compress data sets by capturing inherent correlation patterns.

Conclusions:

• The DCC framework provides a theoretically grounded, widely applicable, and extensible method for data set quality evaluation.
• DCC offers insights into data completeness, representativeness, and potential biases.
• This approach facilitates better data annotation and selection for scientific research and machine learning applications.