Missing Value Estimation using Clustering and Deep Learning within Multiple Imputation Framework

Area of Science:

• Machine Learning
• Data Science
• Statistical Modeling

Background:

• Missing values in tabular data hinder machine learning model performance.
• Multiple Imputations by Chained Equations (MICE) is a popular imputation method using linear conditioning.
• Limitations exist in MICE, especially with high missingness percentages and non-random missing data.

Purpose of the Study:

• To improve imputation accuracy and classification performance of MICE.
• To introduce ensemble learning and deep neural networks (DNN) as replacements for MICE's linear regressors.
• To enhance imputation accuracy further using cluster labels (CISCL).

Main Methods:

• Replaced MICE's linear regressors with ensemble learning and deep neural networks (DNN).
• Incorporated cluster labels (CISCL) derived from training data to characterize samples.
• Conducted extensive analyses on six datasets with up to 80% missing values across three missing types.

Main Results:

• Ensemble learning or DNN within MICE outperformed baseline MICE (b-MICE).
• CISCL significantly improved imputation accuracy, with CISCL + b-MICE outperforming b-MICE universally.
• Proposed DNN-based MICE and gradient boosting MICE plus CISCL (GB-MICE-CISCL) surpassed seven state-of-the-art methods.
• GB-MICE-CISCL enhanced classification accuracy on imputed data across all missingness percentages.
• Identified MICE framework shortcomings at >50% missingness and for non-random missing types.

Conclusions:

• Ensemble learning and DNN integration enhance MICE imputation and downstream classification accuracy.
• CISCL provides a robust method to improve imputation across diverse missing data scenarios.
• GB-MICE-CISCL offers a superior imputation strategy, particularly for complex missing data patterns.
• The study provides a framework for selecting optimal imputation models based on data characteristics.