Mission imputable: Effects of missing data processing on infectious disease detection and prognosis

Area of Science:

• Medical Informatics
• Artificial Intelligence
• Machine Learning

Background:

• Missing data in medical datasets challenges AI/ML pipeline development.
• Inaccurate imputation can lead to biased results and reduced model performance.
• Understanding imputation effects is vital for reliable clinical insights.

Purpose of the Study:

• Systematically investigate imputation method effects on AI/ML performance.
• Analyze clinical implications of different imputation strategies.
• Provide guidance for selecting appropriate imputation techniques.

Main Methods:

• Evaluated six missing data strategies on classification algorithms for medical data.
• Assessed performance using sensitivity and specificity for COVID-19 diagnosis and patient deterioration prediction.
• Conducted feature analysis to understand clinical implications of imputation choices.

Main Results:

• Imputation impact varies by clinical setting; MICE improved sensitivity by 26% in a general cohort.
• Complete-case analysis showed bias in an intensive care cohort due to informative missingness.
• Imputation recovers signals in sparse data and reduces bias in high-acuity settings.

Conclusions:

• Missing data imputation critically impacts AI/ML performance and clinical insights.
• Tailoring imputation techniques to medical data characteristics ensures accurate predictions.
• Findings offer insights for selecting imputation methods in clinical decision-making.