Evaluation methodology for deep learning imputation models.

Area of Science:

• Machine Learning
• Data Science
• Statistical Modeling

Background:

• Deep learning models are increasingly used for imputing missing data in tabular datasets.
• Current evaluation often relies on root mean square error (RMSE), which may not capture all performance aspects.

Purpose of the Study:

• To investigate the limitations of RMSE in evaluating deep learning imputation models.
• To propose and validate a new aggregated metric, reconstruction loss (RL), and an associated evaluation methodology.

Main Methods:

• Comparative analysis of RMSE against alternative statistical metrics (qualitative, predictive accuracy, statistical distance, descriptive statistics).
• Development of the reconstruction loss (RL) metric.
• Evaluation using regression imputation, denoising autoencoders, and generative adversarial nets on healthcare and financial datasets.

Main Results:

• RMSE has limitations in comprehensively evaluating deep learning imputation models.
• The proposed reconstruction loss (RL) metric and methodology effectively assess multiple performance properties.
• The novel methodology demonstrated effectiveness across different model types and datasets.

Conclusions:

• The developed reconstruction loss (RL) metric and evaluation methodology offer a more robust approach to assessing deep learning-based imputation.
• This work provides a valuable tool for researchers and practitioners in data imputation and machine learning.