Using Natural Language Processing to improve EHR Structured Data-based Surgical Site Infection Surveillance.

Area of Science:

• Health Informatics
• Clinical Quality Improvement
• Machine Learning in Healthcare

Background:

• Surgical Site Infection (SSI) surveillance is crucial for patient safety but traditionally relies on manual chart review, leading to high labor costs and underreporting.
• Electronic Health Records (EHRs) offer a potential solution for secondary data use in quality surveillance programs.
• Integrating diverse data sources within EHRs is key to enhancing surveillance capabilities.

Purpose of the Study:

• To evaluate the effectiveness of combining natural language processing (NLP) outputs with structured EHR data for automated Surgical Site Infection (SSI) identification.
• To compare machine learning models that utilize structured data alone versus those incorporating NLP-derived features.

Main Methods:

• Development and evaluation of various machine learning models, including Random Forest classifiers.
• Inclusion of structured EHR data alongside different levels of NLP features: document-level, mention-level, and keyword-based.
• Analysis of model performance using metrics such as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F0.5 score.

Main Results:

• The top-performing model, a Random Forest classifier, incorporated NLP document-level features.
• This enhanced model achieved a sensitivity of 0.58, specificity of 0.97, PPV of 0.54, NPV of 0.98, and an F0.5 score of 0.52.
• Feature contribution analysis and error examination provided insights into model behavior.

Conclusions:

• Integrating NLP outputs with structured EHR data significantly enhances machine learning models for SSI identification.
• This approach offers a more efficient and potentially more accurate alternative to manual surveillance methods.
• Future research should focus on refining NLP techniques and model architectures for broader clinical application.