Related Experiment Video
Updated: Dec 9, 2025
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
[External Validation of Carbapenem-Resistant Enterobacteriaceae Acquisition Risk Prediction Model in a Medium Sized
1Infection Control Unit, Dongeui Medical Center, Busan, Korea.
Purpose:
This study was aimed to evaluate the external validity of a carbapenem-resistant Enterobacteriaceae (CRE) acquisition risk prediction model (the CREP-model) in a medium-sized hospital.
Methods:
This retrospective cohort study included 613 patients (CRE group: 69, no-CRE group: 544) admitted to the intensive care units of a 453-beds secondary referral general hospital from March 1, 2017 to September 30, 2019 in South Korea. The performance of the CREP-model was analyzed with calibration, discrimination, and clinical usefulness.
Results:
The results showed that those higher in age had lower presence of multidrug resistant organisms (MDROs), cephalosporin use ≥ 15 days, Acute Physiology and Chronic Health Evaluation II (APACHE II) score ≥ 21 points, and lower CRE acquisition rates than those of CREP-model development subjects. The calibration-in-the-large was 0.12 (95% CI: - 0.16~0.39), while the calibration slope was 0.87 (95% CI: 0.63~1.12), and the concordance statistic was .71 (95% CI: .63~.78). At the predicted risk of .10, the sensitivity, specificity, and correct classification rates were 43.5%, 84.2%, and 79.6%, respectively. The net true positive according to the CREP-model were 3 per 100 subjects. After adjusting the predictors' cutting points, the concordance statistic increased to .84 (95% CI: .79~.89), and the sensitivity and net true positive was improved to 75.4%. and 6 per 100 subjects, respectively.
Conclusion:
The CREP-model's discrimination and clinical usefulness are low in a medium sized general hospital but are improved after adjusting for the predictors. Therefore, we suggest that institutions should only use the CREP-model after assessing the distribution of the predictors and adjusting their cutting points.
Insights
The carbapenem-resistant Enterobacteriaceae (CRE) prediction model showed low accuracy in a medium-sized hospital. Adjusting the model’s predictors improved its performance, suggesting careful implementation is needed.
Area of Science:
- Infectious Diseases
- Epidemiology
- Healthcare Quality
Background:
- Carbapenem-resistant Enterobacteriaceae (CRE) poses a significant threat in healthcare settings.
- Accurate prediction models are crucial for controlling CRE transmission.
- External validation of existing models is essential to ensure their generalizability.
Purpose of the Study:
- To evaluate the external validity of the carbapenem-resistant Enterobacteriaceae (CRE) acquisition risk prediction model (CREP-model).
- To assess the CREP-model's performance in a medium-sized hospital setting.
- To determine if adjustments to the model could improve its clinical utility.
Main Methods:
- Retrospective cohort study of 613 intensive care unit patients.
- Analysis included calibration, discrimination, and clinical usefulness of the CREP-model.
- Model performance was assessed before and after adjusting predictor cutting points.
Main Results:
- The CREP-model demonstrated low discrimination and clinical usefulness in the studied hospital.
- Calibration slope was 0.87 and concordance statistic was 0.71.
- Adjusting predictor cutting points improved the concordance statistic to 0.84 and sensitivity to 75.4%.
Conclusions:
- The CREP-model's external validity is limited in medium-sized hospitals.
- Adjusting predictor cutting points can enhance the model's performance and clinical usefulness.
- Healthcare institutions should carefully assess and adapt the CREP-model before implementation.
Related Concept Videos
01:13Sensitivity, Specificity, and Predicted Value
Sensitivity is the...
01:22Healthcare Associated Infections II: Preventive Measures
The best practices for preventing healthcare-associated infections include hand hygiene, patient risk...