Prediction modeling using EHR data: challenges, strategies, and a comparison of machine learning approaches

Area of Science:

• Biomedical Informatics
• Machine Learning in Healthcare
• Predictive Analytics

Background:

• Electronic Health Records (EHR) offer extensive patient data.
• Machine learning (ML) methods like Boosting and Support Vector Machines (SVM) show potential for identifying high-risk patients from EHRs.
• Widespread testing of ML for serious conditions like heart failure using EHR data is limited.

Purpose of the Study:

• To develop a model for detecting heart failure more than six months prior to clinical diagnosis using ML on EHR data.
• To evaluate and compare the predictive performance of logistic regression, SVM, and Boosting algorithms.
• To assess the impact of various variable selection methods on heart failure prediction accuracy.

Main Methods:

• A nested case-control study identified Geisinger Clinic primary care patients with EHR data (2001-2006) diagnosed with heart failure.
• Controls were randomly selected and matched for sex, age, and clinic.
• Performance was evaluated using Area Under the Curve (AUC) via 10-fold cross-validation; variable selection methods were compared.

Main Results:

• Logistic regression with Bayesian information criterion selection yielded a parsimonious model (approx. 10 variables) with high AUC (0.77).
• Boosting with a strict variable importance threshold demonstrated comparable performance.
• Support Vector Machines exhibited the lowest performance, potentially due to data imbalance.

Conclusions:

• Heart failure prediction more than six months before diagnosis is feasible using logistic regression and Boosting on EHR data, achieving an AUC of approximately 0.76.
• Effective prediction was possible even with stringent model selection criteria.
• SVM performance was suboptimal, likely impacted by imbalanced datasets.