Developing Clinical Prediction Models Using Primary Care Electronic Health Record Data: The Impact of Data...

Area of Science:

• Health Informatics
• Biostatistics
• Cardiovascular Epidemiology

Background:

• Electronic Health Records (EHR) offer vast data for predictive modeling.
• Data preparation choices can influence model performance and reliability.
• Accurate prediction of cardiovascular events is critical for patient management.

Purpose of the Study:

• To quantify the impact of data preparation choices on prediction model performance using EHR data.
• To evaluate how different data preparation strategies affect the accuracy of cardiovascular event prediction models.

Main Methods:

• Developed Cox proportional hazards models using Dutch primary care EHR data.
• Varied data preparation by adjusting run-in period, outcome definition (diagnosis/medication codes), and missing value imputation (mean/complete case).
• Assessed model performance on a validation set.

Main Results:

• Outcome definition solely on diagnosis codes led to risk underestimation (intercept: 0.84).
• Complete case analysis for missing values resulted in risk overestimation (intercept: -0.52).
• Run-in period length had minimal impact on model calibration and discrimination.

Conclusions:

• Data preparation, particularly outcome definition and missing value handling, substantially affects prediction model calibration.
• These choices can hinder reliable clinical decision support derived from EHR data.
• Transparent reporting of data preparation and modeling choices in EHR studies is essential.