Evaluating Guideline-Adherent Antibiotic Use for Skin Infections Using Natural Language Processing: A Pilot Study

Area of Science:

• Medical Informatics
• Computational Linguistics
• Clinical Decision Support

Background:

• Antibiotic resistance is a growing public health concern.
• Judicious antibiotic use is crucial for effective treatment and preventing resistance.
• Guideline adherence for antibiotic selection in emergency departments (EDs) can be challenging.

Purpose of the Study:

• To develop and validate a natural language processing (NLP) model for assessing antibiotic guideline adherence in ED skin and soft tissue infection (SSTI) cases.
• To evaluate the model's performance against manual physician review.

Main Methods:

• A random forest (RF)/NLP model was developed to classify clinical narratives for methicillin-resistant Staphylococcus aureus (MRSA) versus non-MRSA antibiotic coverage.
• The study included pediatric patients (1-18 years) presenting with SSTIs to the ED.
• Model performance was assessed using sensitivity, specificity, and ROC curve analysis.

Main Results:

• The RF model achieved an AUC of 0.99, with 97.0% sensitivity and 94.9% specificity on the training data.
• On the validation set, the model demonstrated 96.6% sensitivity and 90.1% specificity.
• High performance was maintained despite missing patient history in some clinical narratives.

Conclusions:

• Automated analysis of clinical narratives using NLP is a feasible method for determining antibiotic guideline adherence.
• The NLP model's high performance, even with incomplete data, suggests potential for widespread application in healthcare settings.
• This methodology offers a sustainable approach for improving antibiotic stewardship and patient safety in emergency care.