The impact of imputation quality on machine learning classifiers for datasets with missing values.

Area of Science:

• Machine Learning
• Data Science
• Statistics

Background:

• Classifying samples in incomplete datasets is a common machine learning challenge.
• Real-world datasets often contain missing values requiring imputation before classification.
• Current focus is on optimizing classifier performance after imputation.

Purpose of the Study:

• To evaluate the impact of imputation methods and missingness rates on downstream classifier performance.
• To compare existing imputation quality assessment methods with a novel approach using sliced Wasserstein distance.
• To analyze the stability and interpretability of models trained on imputed data.

Main Methods:

• Utilized three simulated and three real-world clinical datasets with varying missingness patterns.
• Employed Analysis of Variance (ANOVA) to quantify the influence of missingness rate, imputation, and classifier choices.
• Introduced and evaluated discrepancy scores based on sliced Wasserstein distance for imputation quality assessment.

Main Results:

• Classifier performance is highly sensitive to the percentage of missingness in test data.
• Common imputation quality measures often result in data distributions that poorly match the original.
• Novel discrepancy scores demonstrate superior performance in matching underlying data distributions.
• Interpretability of classifier models is compromised when trained on poorly imputed data.

Conclusions:

• The quality of data imputation is critical for downstream classification tasks.
• Poor imputation can lead to considerable negative effects on classifier performance and model interpretability.
• Prioritizing robust imputation quality assessment is imperative for reliable machine learning applications.