Optimising coronary imaging decisions with machine learning: an external validation study
L Malin Overmars1, Bram van Es2, Floor Groepenhoff3
1Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands l.m.overmars-2@umcutrecht.nl.
Insights
Sex-stratified machine learning algorithms using electronic health records (EHRs) show high negative predictive values for excluding coronary stenosis. While promising, further refinement is needed before widespread clinical use.
Area of Science:
- Cardiology
- Medical Informatics
- Machine Learning
Background:
- Diagnosing coronary stenosis is challenging and current methods like CT and angiography are costly and invasive.
- Electronic health records (EHRs) offer a potential non-invasive alternative for excluding coronary stenosis.
- External validation of sex-stratified algorithms is crucial for assessing generalizability across different healthcare settings.
Purpose of the Study:
- To externally validate sex-stratified machine learning algorithms for predicting the absence of coronary stenosis.
- To evaluate algorithm performance in diverse clinical settings using EHR data.
Main Methods:
- Sex-stratified XGBoost algorithms were developed using EHR data from 14,674 patients.
- Algorithms were externally tested on EHR data from 9,252 patients across 13 cardiology centers.
- Absence of coronary stenosis was determined via text mining of radiology reports; performance was measured by negative predictive values (NPVs) and specificities.
Main Results:
- In the training cohort, algorithms achieved NPVs of 0.95 (men) and 0.93 (women) with specificities of 0.14 (men) and 0.26 (women).
- In the testing cohort, NPVs were 0.89 (men) and 0.87 (women), with specificities of 0.07 (men) and 0.18 (women).
- High NPVs were observed across different settings, indicating strong predictive power for the absence of stenosis.
Conclusions:
- Sex-stratified machine learning algorithms using EHR data can non-invasively predict the absence of coronary stenosis with high NPVs.
- The modest specificity suggests limitations for immediate clinical adoption.
- Further research and refinement are necessary before these algorithms can be widely implemented in clinical practice.
Background:
Exclusion of coronary stenosis in individuals with suggestive symptoms is challenging. Cardiac CT or coronary angiography is often used but is inefficient and costly and involves risks. Sex-stratified algorithms based on electronic health records (EHRs) could be a non-invasive alternative for excluding coronary stenosis, yet their performance may vary by healthcare settings. Thus, external validation is crucial for determining their generalisability. This study aimed to externally validate sex-stratified machine learning algorithms based on EHR data to predict the absence of coronary stenosis, evaluated in diverse clinical settings.
Methods:
Sex-stratified XGBoost algorithms were trained on EHR data from patients who underwent coronary imaging at the University Medical Center Utrecht (n=14 674) and externally tested on EHR data of 13 Cardiology centres in the Netherlands (n=9252). The outcome was defined as the absence of coronary stenosis, identified through text mining of radiology report conclusions, and predictive performance was assessed by negative predictive values (NPVs) and specificities.
Results:
On the training cohort (9298 men (median age 55 years, 73% no coronary stenosis) and 5376 women (median age 59 years, 83% no coronary stenosis)), the algorithms showed NPVs and specificities of 0.95 and 0.14 in men and 0.93 and 0.26 in women, respectively. On the testing cohort (4762 men (median age 60 years, 60% no coronary stenosis) and 4490 women (median age 60 years, 83% no coronary stenosis)), the algorithm showed NPVs and specificities of 0.89 and 0.07 in men and 0.87 and 0.18 in women, respectively.
Conclusions:
This study externally validates sex-stratified machine learning algorithms using EHR data to non-invasively predict the absence of coronary stenosis, with high NPVs observed across settings. However, given the modest specificity and study limitations, these findings should be considered preliminary, warranting further refinement before clinical adoption.
