Related Experiment Video
Updated: Sep 29, 2025
A Neonatal Imaging Model of Gram-Negative Bacterial Sepsis
Published on: August 12, 2020
Development and validation of machine learning-driven prediction model for serious bacterial infection among febrile
Bongjin Lee1, Hyun Jung Chung2, Hyun Mi Kang3
1Department of Pediatrics, Seoul National University Hospital, Seoul, Korea.
Insights
A new machine learning model accurately predicts serious bacterial infections (SBI) in febrile children, even with missing data. This AI tool offers superior performance over traditional methods for diagnosing critical pediatric conditions.
Area of Science:
- Pediatric Infectious Diseases
- Machine Learning in Healthcare
- Clinical Decision Support Systems
Background:
- Serious bacterial infections (SBI) in children, including meningitis and sepsis, are life-threatening conditions requiring accurate and timely diagnosis.
- Existing SBI prediction tools often fail when crucial data points are missing, limiting their clinical utility.
- The need for robust diagnostic aids that can function effectively despite incomplete patient information is critical in emergency settings.
Purpose of the Study:
- To develop and validate a machine learning (ML)-driven model for predicting SBI in febrile children.
- To create a model capable of robust prediction even when patient data values are missing.
- To compare the performance of the ML model against conventional methods in both internal and external validation cohorts.
Main Methods:
- A multicenter retrospective observational study involving febrile children under 6 years old across three tertiary hospitals (2016-2018).
- A derivation cohort (two hospitals) was used to train a Random Forest (RF) ML model, while a third hospital's data served as an external validation cohort.
- Performance was evaluated using Area Under the Receiver Operating Characteristic Curve (AUROC) and Area Under the Precision-Recall Curve (AUPRC), comparing RF to a Conventional Logistic Regression (CLR) model.
Main Results:
- In the derivation cohort, the RF model achieved an AUROC of 0.964 and AUPRC of 0.753, significantly outperforming the CLR model (AUROC 0.902, AUPRC 0.573).
- External validation in the third hospital confirmed the RF model's superior performance with an AUROC of 0.950 and AUPRC of 0.605, compared to CLR (AUROC 0.815, AUPRC 0.586).
- The ML-driven RF model demonstrated robust performance and effectiveness despite the presence of missing data points in patient records.
Conclusions:
- A novel machine learning-driven prediction model for SBI in febrile children has been successfully developed and validated.
- The developed model exhibits superior predictive performance compared to conventional methods, particularly in handling missing data.
- This AI-powered tool offers a promising advancement for improving the accuracy and reliability of SBI diagnosis in pediatric emergency departments.
Abstract:
Serious bacterial infection (SBI) in children, such as bacterial meningitis or sepsis, is an important condition that can lead to fatal outcomes. Therefore, since it is very important to accurately diagnose SBI, SBI prediction tools such as 'Refined Lab-score' or 'clinical prediction rule' have been developed and used. However, these tools can predict SBI only when there are values of all factors used in the tool, and if even one of them is missing, the tools become useless. Therefore, the purpose of this study was to develop and validate a machine learning-driven model to predict SBIs among febrile children, even with missing values. This was a multicenter retrospective observational study including febrile children <6 years of age who visited Emergency departments (EDs) of 3 different tertiary hospitals from 2016 to 2018. The SBI prediction model was trained with a derivation cohort (data from two hospitals) and externally tested with a validation cohort (data from a third hospital). A total of 11,973 and 2,858 patient records were included in the derivation and validation cohorts, respectively. In the derivation cohort, the area under the receiver operating characteristic curve (AUROC) of the RF model was 0.964 (95% confidence interval [CI], 0.943-0.986), and the area under the precision-recall curve (AUPRC) was 0.753 (95% CI, 0.681-0.824). The conventional LR (CLR) model showed corresponding values of 0.902 (95% CI, 0.894-0.910) and 0.573 (95% CI, 0.560-0.586), respectively. In the validation cohort, the AUROC (95% CI) of the RF model was 0.950 (95% CI, 0.945-0.956), the AUPRC was 0.605 (95% CI, 0.593-0.616), and the CLR presented corresponding values of 0.815 (95% CI, 0.789-0.841) and 0.586 (95% CI, 0.553-0.619), respectively. We developed a machine learning-driven prediction model for SBI among febrile children, which works robustly despite missing values. And it showed superior performance compared to CLR in both internal validation and external validation.
