Related Experiment Video
Updated: May 8, 2025
Author Spotlight: Advancing CBCT and Digital Dental Image Integration with AI-Assisted Digitization
Published on: February 23, 2024
Development and validation of artificial intelligence models for early detection of postoperative infections
Siri L van der Meijden1,2, Anna M van Boekel1, Laurens J Schinkelshoek2
1Intensive Care Unit, Leiden University Medical Centre, Leiden, the Netherlands.
Background:
Postoperative infections significantly impact patient outcomes and costs, exacerbated by late diagnoses, yet early reliable predictors are scarce. Existing artificial intelligence (AI) models for postoperative infection prediction often lack external validation or perform poorly in local settings when validated. We aimed to develop locally valid models as part of the PERISCOPE AI system to enable early detection, safer discharge, and more timely treatment of patients.
Methods:
We developed and validated XGBoost models to predict postoperative infections within 7 and 30 days of surgery. Using retrospective pre-operative and intra-operative electronic health record data from 2014 to 2023 across various surgical specialities, the models were developed at Hospital A and validated and updated at Hospitals B and C in the Netherlands and Belgium. Model performance was evaluated before and after updating using the two most recent years of data as temporal validation datasets. Main outcome measures were model discrimination (area under the receiver operating characteristic curve (AUROC)), calibration (slope, intercept, and plots), and clinical utility (decision curve analysis with net benefit).
Findings:
The study included 253,010 surgical procedures with 23,903 infections within 30-days. Discriminative performance, calibration properties, and clinical utility significantly improved after updating. Final AUROCs after updating for Hospitals A, B, and C were 0.82 (95% confidence interval (CI) 0.81-0.83), 0.82 (95% CI 0.81-0.83), and 0.91 (95% CI 0.90-0.91) respectively for 30-day predictions on the temporal validation datasets (2022-2023). Calibration plots demonstrated adequate correspondence between observed outcomes and predicted risk. All local models were deemed clinically useful as the net benefit was higher than default strategies (treat all and treat none) over a wide range of clinically relevant decision thresholds.
Interpretation:
PERISCOPE can accurately predict overall postoperative infections within 7- and 30-days post-surgery. The robust performance implies potential for improving clinical care in diverse clinical target populations. This study supports the need for approaches to local updating of AI models to account for domain shifts in patient populations and data distributions across different clinical settings.
Funding:
This study was funded by a REACT EU grant from European Regional Development Fund (ERDF) and Kansen voor West.
Insights
This study developed an AI system, PERISCOPE, to accurately predict postoperative infections early. Local model updating ensures reliable predictions, improving patient care and timely treatment.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Healthcare
- Surgical Outcomes Research
Background:
- Postoperative infections pose significant risks to patient outcomes and healthcare costs.
- Early detection of infections is crucial but hindered by a lack of reliable predictors.
- Existing artificial intelligence (AI) models often lack external validation and local applicability.
Purpose of the Study:
- To develop and validate locally applicable AI models for early prediction of postoperative infections.
- To enhance the PERISCOPE AI system for safer patient discharge and timely treatment initiation.
- To address the limitations of existing AI models in diverse clinical settings.
Main Methods:
- Development and validation of XGBoost models using retrospective electronic health record data (2014-2023).
- Models were trained at Hospital A and validated/updated at Hospitals B and C.
- Performance evaluated using area under the receiver operating characteristic curve (AUROC), calibration, and decision curve analysis.
Main Results:
- The study analyzed 253,010 procedures, identifying 23,903 infections within 30 days.
- Model performance, calibration, and clinical utility significantly improved after local updating.
- Post-update 30-day prediction AUROCs reached 0.82 (Hospital A), 0.82 (Hospital B), and 0.91 (Hospital C).
Conclusions:
- The PERISCOPE AI system accurately predicts postoperative infections within 7 and 30 days.
- Robust local model updating is essential to maintain AI performance across different clinical settings and patient populations.
- This approach holds potential for improving clinical care and patient outcomes in diverse healthcare environments.
Related Concept Videos
01:29Issues And Trends In Healthcare Delivery System
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
01:22Healthcare Associated Infections II: Preventive Measures
The best practices for preventing healthcare-associated infections include hand hygiene, patient risk...
01:30Methods of Documentation VII: EMR
01:03Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
01:30Health Information Technology and Healthcare Information System
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
01:30Current Trends in Nursing II