Related Experiment Video
Updated: Jul 4, 2026
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
Development and External Validation of a Machine Learning Model for 10-Year Ischemic Stroke Risk Prediction in
Ahmed Khattab1, Zhe Wang1,2, Vinodh Srinivasasainagendra3
1Department of Epidemiology, School of Public Health, The University of Alabama at Birmingham, Birmingham, AL, USA.
Importance:
Machine-learning models for ischemic stroke risk prediction are rarely validated across ancestrally distinct cohorts, and the contributions of polygenic risk scores (PRS) and self-reported race in such models remain unclear.
Objective:
To develop and externally validate a 10-year ischemic stroke risk model and quantify the incremental contributions of laboratory trajectories, PRS, and self-reported race and ethnicity across populations.
Design Setting And Participants:
Retrospective cohort study with model development in the All of Us (AoU) Research Program (n = 34,987; 1,920 incident strokes) and external validation in the BioMe Biobank at Mount Sinai (n = 10,693; 107 incident strokes). Adults aged ≥45 years with ≥1 year of pre-baseline electronic health record data were anchored to a January 2010 baseline with 10-year follow-up.
Exposures:
Three XGBoost model tiers added laboratory feature trajectories (M2) and 20 PRS (M3) to clinical baseline features (M1); evaluated under race-blind and race-aware specifications.
Main Outcomes And Measures:
First inpatient ischemic stroke within 10 years; discrimination (area under the receiver operating characteristic curve [AUROC]) and calibration (observed-to-expected [O/E] ratio).
Results:
In the AoU test partition (n = 6,998; 384 cases), M3 achieved AUROC 0.813 (95% CI, 0.788-0.837), outperforming the Revised Framingham Stroke Risk Profile (ΔAUROC 0.164) and Pooled Cohort Equations (ΔAUROC 0.181; both P < 0.001). Discrimination transferred to BioMe (AUROC 0.745) but predictions were systematically high (aggregate O/E 0.12 vs. 1.00 in AoU), consistent with intercept-shift miscalibration; BioMe-fit intercept recalibration restored calibration in African American and Hispanic but not European American participants. PRS contribution was significant only in the BioMe Hispanic (ΔAUROC + 0.042; P = 0.003), with no significant within-stratum gain in five other cohort-by-race combinations. Adding self-reported race produced small gains when combined with PRS (BioMe + 0.022, P = 0.034; AoU + 0.006, P = 0.052) but not when added without PRS.
Conclusions And Relevance:
A machine-learning ensemble combining clinical, laboratory, and polygenic features outperformed traditional risk scores by 0.16-0.18 AUROC and retained discriminative validity in an ancestrally distinct external cohort but required site-specific recalibration of absolute risk. The marginal contribution of self-reported race overlapped with polygenic signal, supporting per-ancestry calibration over universal race-aware model deployment.