Flexible imputation toolkit for electronic health records

Area of Science:

• Biomedical Informatics
• Data Science
• Health Data Analysis

Background:

• Missing data in electronic health records (EHRs) hinders accurate analysis.
• Developing robust imputation methods is crucial for leveraging EHR data.
• Existing imputation techniques may not adapt to diverse data characteristics.

Purpose of the Study:

• Introduce Pympute, a Python package for efficient and robust missing value imputation in EHRs.
• Evaluate the performance of Pympute's Flexible algorithm against existing imputation methods.
• Investigate the impact of data characteristics like skewness on imputation algorithm selection.

Main Methods:

• Developed Pympute, featuring the Flexible algorithm for adaptive imputation.
• Benchmarked ten machine learning imputation algorithms against Flexible using real-world EHR laboratory data.
• Implemented data simulation to generate realistic datasets for controlled evaluation.
• Analyzed the influence of missingness and skewness on Flexible's algorithm selection.

Main Results:

• Pympute's Flexible method demonstrated significantly improved imputation performance over single-model approaches.
• Data simulation based solely on covariance did not replicate real-world imputation algorithm selection behavior.
• Data skewness prompted the Flexible algorithm to favor nonlinear imputation models.
• Flexible's adaptive selection enhances imputation accuracy for EHR datasets.

Conclusions:

• Pympute provides a versatile and user-friendly solution for missing data challenges in EHRs.
• Considering data distribution patterns, particularly skewness, is vital for optimal imputation.
• The Flexible algorithm offers a superior approach to missing value imputation in complex health datasets.