Related Experiment Videos
Automated Identification of Surgical Site Infections From Electronic Medical Records: Retrospective Observational
Arjun Chakraborty1, Peter Tarczy-Hornoch1,2,3, Dustin Long4
1Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington, 222 15th Street SW, Rochester, MN, 55902, United States, 1 5107095904.
Background:
Surgical site infections (SSIs) affect 160,000 to 300,000 patients annually, increasing postoperative mortality, causing significant complications, and incurring US $3.5 to US $10 billion in excess costs each year. Effective SSI surveillance can inform strategies to mitigate these outcomes. Traditional SSI surveillance methods, primarily manual chart reviews, are costly and labor-intensive.
Objective:
This study aimed to evaluate whether an automated SSI surveillance system built using newer natural language processing methods and deep learning could outperform previous approaches and whether such an approach could enable more efficient infection surveillance.
Methods:
Our dataset comprised approximately 30,000 surgical cases from the University of Washington Medical Center (UWMC) and Harborview Medical Center (HMC). Data from UWMC were captured for the National Surgical Quality Improvement Program, and data from HMC were captured for the National Healthcare Safety Network. Electronic health record (EHR) data for each surgical case included structured data pertaining to surgical procedure characteristics, laboratory values, and antibiotic administration, as well as clinical text notes for a surgical case from 7 days before to 90 days after surgery. Using this data, we applied a myriad of machine learning approaches to the task of SSI prediction. We reported the following performance metrics: F1-score, precision (positive predictive value), recall (sensitivity), area under the precision-recall curve, and precision at 0.9 recall for each machine learning approach.
Results:
In a cohort of 5996 surgical cases, incorporating multimodal EHR information-including contextual information from clinical text and temporal information from laboratory values-improved SSI prediction performance. Models using structured data and clinical text outperformed structured data alone (F1=0.68, 95% CI 0.68-0.69 vs F1=0.55, 95% CI 0.54-0.56; P<.001). Adding temporal features further improved performance (F1=0.70, 95% CI 0.69-0.71; P<.001). Deep learning approaches leveraging large language models also outperformed the state-of-the-art rule-based system (F1=0.70, 95% CI 0.69-0.71 vs F1=0.43; P<.001). The optimal approach combined foundation models for text summarization with deep learning methods for clinical text and temporal data processing. This system achieved a precision of 0.38 at 0.9 recall, demonstrating its potential for efficient, data-driven SSI surveillance.
Conclusions:
Automated surveillance approaches-particularly deep learning approaches-in combination with voluminous, multimodal data from the EHR, can enable more efficient infection surveillance processes. This has the potential to increase the quantity of SSI surveillance data available to guide interventions aimed at reducing SSI rates.
