Development, Validation, and Comparison of Machine Learning Models for Predicting Pediatric Surgical Site Infections
Carrie T Chan1,2, Mark J Pletcher1, Karthik Balakrishnan3
1From the Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, CA (Chan, Pletcher, Hswen, Scheffler).
Insights
Machine learning models can predict surgical site infections (SSIs) in children. A regularized logistic regression model shows promise for clinical integration to personalize SSI risk and prevention strategies.
Area of Science:
- Pediatric Surgery
- Infectious Disease Epidemiology
- Health Informatics
Background:
- Surgical site infections (SSIs) significantly increase pediatric postoperative morbidity.
- Rising SSI rates contrast with their largely preventable nature.
- Existing SSI prediction models are primarily designed for adult populations, necessitating pediatric-specific tools.
Purpose of the Study:
- To develop, validate, and compare machine learning (ML) models for predicting pediatric SSI risk.
- To identify a suitable ML model for seamless integration into clinical workflows.
- To enhance individualized risk assessment and targeted infection prevention strategies for pediatric patients.
Main Methods:
- Retrospective cohort analysis of over 1.1 million pediatric cases from the NSQIP-P database (2012-2022).
- Development and comparison of five ML models: logistic regression, random forest, gradient boosted trees, k-nearest neighbors, and neural networks.
- Performance evaluation using Brier scores, c-statistics, and calibration metrics, with bootstrap resampling for confidence intervals.
Main Results:
- All developed ML models demonstrated comparable performance in predicting pediatric SSIs.
- Regularized logistic regression exhibited strong predictive accuracy (c-statistic 0.77, Brier score 0.023) and was selected for its clinical feasibility.
- Key predictors of SSI risk included procedural codes, diagnoses, comorbidities, acuity markers, laboratory values, and patient demographics.
Conclusions:
- Machine learning models effectively predict pediatric SSI risk utilizing preoperative data.
- The proposed regularized logistic regression model is a strong candidate for EHR integration, enabling personalized SSI risk estimation.
- Future work includes external validation and user-centered implementation studies to facilitate clinical adoption.
Background:
Surgical site infections (SSIs) cause substantial postoperative morbidity in children. Despite being largely preventable, SSI rates have continued to rise. Existing SSI predictive models are predominantly designed for adults. This study aimed to develop, validate, and compare machine learning models for predicting pediatric SSI risk and recommend a model for clinical workflow integration.
Study Design:
We conducted a retrospective cohort analysis of 1,152,034 cases from the NSQIP-Pediatrics (NSQIP-P) database (2012 to 2022), including 27,562 SSIs. Data were split into training (1,101,950) and test (50,084) sets. We developed 5 models: elastic net-regularized logistic regression, random forest, gradient boosted trees, k-nearest neighbors, and neural networks. Performance was evaluated using Brier scores, c-statistics, and calibration metrics, with bootstrap resampling to estimate confidence intervals.
Results:
All models performed similarly (Brier scores 0.023 to 0.024; c-statistics 0.72 to 0.77). Regularized logistic regression showed strong performance (Brier score 0.023; c-statistic 0.77) and was selected for its balance of predictive accuracy, computational efficiency, and feasibility for clinical use. Key predictors included procedural codes, perioperative diagnoses, comorbidities, acuity markers, labs, and patient demographics. In the test set, mean predicted SSI risk was 2.4%; 62% of patients had risk <2%, whereas 3% had risk ≥10%.
Conclusions:
Machine learning models can predict pediatric SSI risk using preoperative data. Our regularized logistic regression model is a promising candidate for integration into electronic health record systems to enable individualized risk estimation and targeted infection prevention. Next steps include external validation and user-centered implementation studies.
