Preserving Informative Presence: How Missing Data and Imputation Strategies Affect the Performance of an AI-Based...

Area of Science:

• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems
• Predictive Analytics

Background:

• Data availability significantly impacts the performance of AI-based early warning scores (EWSs).
• The VitalCare-Major Adverse Event Score (VC-MAES) is an AI-based EWS using imputation for missing values.
• Evaluating imputation strategies is crucial for reliable AI-driven clinical deterioration prediction.

Purpose of the Study:

• To assess how missing data extent and imputation methods affect the predictive performance of the VC-MAES.
• To compare VC-MAES performance against traditional EWSs under various data scenarios.
• To determine optimal data handling strategies for AI-based EWSs.

Main Methods:

• Analysis of 6039 patient encounters from Keimyung University Dongsan Hospital.
• Evaluation of VC-MAES performance with: 1) vital signs and age only, 2) full clinical variables, 3) mean imputation, and 4) Multiple Imputation by Chained Equations (MICE).
• Comparison of Area Under the Receiver Operating Characteristic Curve (AUROC) against traditional EWSs.

Main Results:

• VC-MAES achieved an AUROC of 0.896 with limited data (vitals, age), surpassing traditional EWSs (NEWS: 0.797, MEWS: 0.722).
• Full clinical data improved VC-MAES AUROC to 0.918.
• Mean imputation (0.885) and MICE (0.827) resulted in lower performance compared to default imputation.

Conclusions:

• VC-MAES demonstrates robust predictive capabilities, outperforming traditional EWSs even with minimal input.
• Actual clinical data integration significantly enhances AI-based EWS accuracy.
• Imputation strategies like mean imputation or MICE may reduce performance, highlighting the need to consider missingness patterns and context.