Evaluation of a machine learning capability for a clinical decision support system to enhance antimicrobial...

Antimicrobial stewardship programs have been shown to limit the inappropriate use of antimicrobials. Hospitals are increasingly relying on clinical decision support systems to assist in the demanding prescription reviewing process. In previous work, we have reported on an emerging clinical decision support system for antimicrobial stewardship that can learn new rules supervised by user feedback. In this paper, we report on the evaluation of this system.

The evaluated system uses a knowledge base coupled with a supervised learning module that extracts classification rules for inappropriate antimicrobial prescriptions using past recommendations for dose and dosing frequency adjustments, discontinuation of therapy, early switch from intravenous to oral therapy, and redundant antimicrobial spectrum. Over five weeks, the learning module was deployed alongside the baseline system to prospectively evaluate its ability to discover rules that complement the existing knowledge base for identifying inappropriate prescriptions of piperacillin-tazobactam, a frequently used antimicrobial.

The antimicrobial stewardship pharmacists reviewed 374 prescriptions, of which 209 (56% of 374) were identified as inappropriate leading to 43 recommendations to optimize prescriptions. The baseline system combined with the learning module triggered alerts in 270 prescriptions with a positive predictive value of identifying inappropriate prescriptions of 74%. Of these, 240 reviewed prescriptions were identified by the alerts of the baseline system with a positive predictive value of 82% and 105 reviewed prescriptions were identified by the alerts of the learning module with a positive predictive value of 62%. The combined system triggered alerts for all 43 recommendations, resulting in a rate of actionable alerts of 16% (43 recommendations of 270 reviewed alerts); the baseline system triggered alerts for 38 interventions, resulting in a rate of actionable alerts of 16% (38 of 240 reviewed alerts); and the learning module triggered alerts for 17 interventions, resulting in a rate of actionable alerts of 16% (17 of 105 reviewed alerts). The learning module triggered alerts for every inappropriate prescription missed by the knowledge base of the baseline system (n=5).

The learning module was able to extract clinically relevant rules for multiple types of antimicrobial alerts. The learned rules were shown to extend the knowledge base of the baseline system by identifying pharmacist interventions that were missed by the baseline system. The learned rules identified inappropriate prescribing practices that were not supported by local experts and were missing from its knowledge base. However, combining the baseline system and the learning module increased the number of false positives.