Missing value imputation in high-dimensional phenomic data: imputable or not, and how?

Area of Science:

• Biomedical research
• Bioinformatics
• Data science

Background:

• Phenomic data, crucial for complex disease research, often contains missing values (MVs).
• Existing imputation methods are primarily designed for continuous microarray data, limiting their application to diverse phenomic data types (continuous, nominal, binary, ordinal).
• A comprehensive guideline for phenomic missing data imputation is lacking.

Purpose of the Study:

• To investigate existing imputation methods for phenomic data.
• To propose a self-training selection (STS) scheme for optimal method selection.
• To introduce an "imputability measure" (IM) to identify un-imputable missing values.

Main Methods:

• Developed four K-nearest-neighbor (KNN) variations: KNN-V, KNN-S, KNN-H, and KNN-A.
• Compared KNN variations with existing methods: multivariate imputation by chained equations (MICE) and missForest.
• Utilized simulations and three lung disease phenomic datasets for evaluation.
• Implemented the STS scheme for method selection and evaluated its accuracy.

Main Results:

• Multivariate imputation by chained equations (MICE) showed suboptimal performance.
• KNN-A, KNN-H, and random forest were identified as top-performing imputation methods.
• Imputing values with low imputability measures significantly increased errors and could harm downstream analyses.
• The STS scheme demonstrated accuracy in selecting the best imputation method.

Conclusions:

• No single imputation method universally excels across all phenomic datasets.
• The "imputability measure" is critical for identifying and handling problematic missing values.
• The STS scheme provides a reliable approach for selecting appropriate imputation methods.
• An R package "phenomeImpute" is available for practical applications.