Applying Machine Learning Across Sites: External Validation of a Surgical Site Infection Detection Algorithm

Area of Science:

• Medical Informatics
• Machine Learning in Healthcare
• Surgical Quality Improvement

Background:

• Surgical complications incur significant costs and consequences.
• Manual chart review for complication detection is time-consuming and inefficient.
• Automated methods are needed to streamline quality improvement processes.

Purpose of the Study:

• To assess the generalizability of machine learning algorithms for surgical site infection (SSI) detection across different institutions.
• To validate automated SSI detection models developed at one center using data from a distinct center.

Main Methods:

• Electronic health record (EHR) data from two distinct centers (Site A and Site B) were used.
• Machine learning models were developed and internally validated at Site A, then externally validated at Site B.
• Models focused on detecting superficial, organ/space, and total SSIs within 30 days postoperatively, validated using Area Under the Curve (AUC) scores.

Main Results:

• AUC scores for SSI detection were not statistically different between the two centers for all outcomes (superficial, organ/space, total SSI).
• External validation demonstrated the generalizability of the developed machine learning algorithms.
• Decreased false negative rates were observed with increased case review volume, suggesting potential for focused chart review.

Conclusions:

• Machine learning algorithms for SSI detection are generalizable across different institutions.
• Automated SSI detection models can be practically applied to accelerate and focus chart review processes.
• This technology supports efficient surgical quality improvement initiatives.