Development and prospective evaluation of a machine learning model to predict vomiting among pediatric cancer and
Adam Paul Yan1,2, Lin Lawrence Guo2, Priya Patel1,3
1Division of Haematology/Oncology, The Hospital for Sick Children, 555 University Avenue, Toronto, ON, M5G1X8, Canada.
Insights
A machine learning model using electronic health records can predict vomiting risk in pediatric oncology and hematopoietic cell transplant patients. This model showed consistent performance in a prospective trial, paving the way for clinical integration.
Area of Science:
- Machine learning applications in healthcare
- Predictive modeling in oncology
- Electronic health record data utilization
Background:
- Vomiting is a common side effect for pediatric patients undergoing oncology and hematopoietic cell transplant (HCT) treatments.
- Accurate prediction of vomiting risk is crucial for timely intervention and improved patient care.
Purpose of the Study:
- To develop a machine learning (ML) model using electronic health record (EHR) data to predict vomiting risk within 96 hours for pediatric oncology and HCT patients.
- To prospectively evaluate the developed ML model's performance in a silent trial.
Main Methods:
- Utilized retrospective EHR data from SEDAR (2018-2024) and conducted a prospective silent trial (2024).
- Trained and compared L2-regularized logistic regression, LightGBM, and XGBoost models.
- The outcome was defined as any vomiting within 96 hours post-prediction time.
Main Results:
- The best-performing model, LightGBM trained on all inpatients, achieved an AUROC of 0.730 retrospectively and 0.716 prospectively.
- The model utilized 2,859 features and was evaluated on 7,408 retrospective and 340 prospective admissions.
- The ML model demonstrated satisfactory predictive performance in both retrospective and prospective phases.
Conclusions:
- EHR data can effectively train ML models to predict vomiting in pediatric oncology and HCT inpatients.
- The developed ML model maintained its predictive accuracy in a prospective silent trial.
- Future work will focus on clinical workflow integration and assessing the model's impact on vomiting control.
Purpose:
Objectives were to develop a machine learning (ML) model based on electronic health record (EHR) data to predict the risk of vomiting within a 96-hour window after admission to the pediatric oncology and hematopoietic cell transplant (HCT) services using retrospective data and to evaluate the model prospectively in a silent trial.
Patients And Methods:
Admissions between 2018-06-02 to 2024-02-17 (retrospective) and 2024-05-09 to 2024-08-05 (prospective) to the oncology or HCT services were included. Data source was SEDAR, a curated and validated approach to deliver EHR data for ML. Prediction time was 08:30 the morning following admission. The outcome was any vomiting within 96 h following prediction time. We trained models using L2-regularized logistic regression, LightGBM and XGBoost. Training cohorts include the target cohort and all inpatient admissions.
Results:
There were 7,408 admissions in the retrospective phase and 340 admissions in the prospective silent trial phase. The best-performing model in the retrospective phase was the LightGBM model trained on all inpatients. The number of features in the final model was 2,859. The area-under-the-receiver-operating-characteristic curve (AUROC) was 0.730 (95% confidence interval (CI) 0.694-0.765) for the retrospective phase and 0.716 (95% CI 0.649-0.784) for the prospective silent trial phase.
Conclusions:
We found that data in the EHR could be used to develop a retrospective ML model to predict vomiting among pediatric oncology and HCT inpatients. This model retained satisfactory performance in a prospective silent trial. Future plans will include deployment into clinical workflows and determining if the model improves vomiting control.
Related Concept Videos
01:22Pathophysiology of Vomiting
02:43Mouse Models of Cancer Study
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...