A Pragmatic Ensemble Strategy for Missing Values Imputation in Health Records

Area of Science:

• Medical Informatics
• Data Science
• Machine Learning

Background:

• Medical data frequently contains missing values, impacting the reliability of computer modeling for clinical decision support.
• Missing data can occur in both training and testing datasets, complicating predictive accuracy.

Purpose of the Study:

• To evaluate and propose an effective imputation strategy for handling missing values in healthcare datasets.
• To develop an ensemble imputation model that selects the optimal imputation method based on data characteristics.

Main Methods:

• An ensemble imputation model combining mean, k-nearest neighbor, and iterative imputation was developed.
• The model dynamically selects imputation strategies based on attribute correlations within missing value features.
• Performance was evaluated using eXtreme gradient boosting, random forest, and support vector regressors on real-world healthcare data.

Main Results:

• The proposed ensemble imputation strategy demonstrated superior accuracy compared to standard imputation methods and data deletion.
• Experiments included simulations with varying missing data frequencies to assess robustness.
• The ensemble approach effectively addressed missing values in both training and testing datasets.

Conclusions:

• The developed Ensemble Strategy for Missing Value (ESMV) provides an accurate and unbiased method for analyzing healthcare data with missing values.
• This approach enhances the reliability of statistical modeling and medical decision-making.
• The ESMV offers a significant improvement over existing missing data handling techniques.