The performance of prognostic models depended on the choice of missing value imputation algorithm: a simulation study

Area of Science:

• Biostatistics
• Machine Learning in Healthcare
• Clinical Prediction Modeling

Background:

• Missing predictor values frequently hinder the development of robust clinical prediction models.
• Existing imputation methods vary in complexity, from simple single imputation to advanced multiple imputation techniques using chained equations.
• Machine learning algorithms and flexible modeling are increasingly integrated into imputation strategies.

Purpose of the Study:

• To evaluate the comparative performance of different missing value imputation methods in the context of clinical prediction model development.
• To determine if specific imputation algorithms consistently outperform others across various performance metrics.

Main Methods:

• Simulated development and validation cohorts mimicking real data distributions.
• Applied three R-based imputation algorithms: mice, aregImpute, and missForest, under 36 missingness scenarios.
• Assessed model performance using Brier score, c-statistic, calibration, and prediction error, comparing against full data analysis.

Main Results:

• No imputation method fully replicated the performance of complete data; complete case analysis performed worst.
• aregImpute and mice (with 100 imputations) demonstrated the highest predictive accuracy across scenarios.
• aregImpute notably achieved calibration slopes close to one, outperforming even full data analysis in this aspect.

Conclusions:

• Model calibration is more sensitive to imputation method choice than discrimination.
• Multiple imputation methods, particularly mice and aregImpute, are robust and recommended for handling missing data in prediction models.
• These methods effectively manage linear and nonlinear predictor-outcome associations, providing reliable results.