Development and temporal validation of a machine-learning model for vasovagal reactions in whole-blood donors: a large retrospective cohort study

Vasovagal reactions (VVRs) are common systemic adverse events associated with blood donation. Evidence remains limited on whether VVR prediction models retain performance in future donor populations, remain calibrated over time, and support practical risk stratification. We developed a machine-learning model using routinely available donation information and longitudinal donation history and evaluated its temporal transportability and risk-stratification performance.

We analyzed 5,310,116 whole-blood donation encounters recorded by the Blood Center of Zhejiang Province from January 2018 through June 2026. Donation encounters from 2018-2023 were used for development, 2024 for held-out within-center temporal validation, and January 2025-June 2026 for the final held-out temporal test. Six candidate models were compared using 20 predictors available before donation and expanding-window forward temporal validation. Average precision (AP) was the primary model-selection metric, with ROC-AUC as a complementary measure. The final model, calibration function, and risk thresholds were fixed during development and remained unchanged in subsequent cohorts.

XGBoost was selected under the prespecified AP-first, ROC-AUC-supported framework. In the 2024 temporal validation cohort, AP was 0.0336 and ROC-AUC was 0.7776. The final temporal test included 1,014,652 donation encounters and 5,957 VVRs (0.587%); AP was 0.0338 (95% CI, 0.0305-0.0374) and ROC-AUC was 0.7633 (95% CI, 0.7572-0.7688). After applying the fixed Platt calibration function, the observed-to-expected event ratio was 1.169. The fixed primary alert threshold flagged 1.55% of encounters, among which the VVR rate was 3.45%, 5.88 times the overall rate. Performance was better among encounters with prior observed donation history than among those without such history (ROC-AUC, 0.7713 vs 0.6822).

A temporally validated XGBoost model based on routine donation information and longitudinal history identified a small VVR risk-enriched group in future cohorts. Discrimination and calibration changed over time, and performance was weaker when prior observed donation history was unavailable. The model may support donor-site risk stratification and targeted observation, but multicenter external validation and prospective implementation studies are required before routine use.