Machine Learning Approach to Reduce Alert Fatigue Using a Disease Medication-Related Clinical Decision Support...

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Health Informatics

Background:

• Computerized Physician Order Entry (CPOE) systems integrated with Clinical Decision Support Systems (CDSSs) aim to minimize medication errors and enhance patient safety.
• CDSSs generate automatic alerts to guide physicians' clinical decisions and influence prescribing habits.
• High rates of alert overrides (90%-96%) by physicians indicate a significant challenge with alert fatigue, prompting research into effective mitigation strategies.

Purpose of the Study:

• To develop machine learning (ML) prediction models for forecasting physician responses to alerts from disease medication-related CDSSs.
• The goal is to mitigate alert fatigue experienced by healthcare professionals.

Main Methods:

• Data were collected from a disease medication-related CDSS at a Taiwanese university teaching hospital, focusing on alerts triggered between August 2018 and May 2019.
• Various ML models, including Artificial Neural Network (ANN), Random Forest (RF), Naïve Bayes (NB), Gradient Boosting (GB), and Support Vector Machine (SVM), were employed.
• The dataset was divided into 80% for training and 20% for testing.

Main Results:

• The Artificial Neural Network (ANN) model achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.94 and an accuracy of 0.85.
• Other models showed varying performance: RF (AUROC 0.93), NB (AUROC 0.91), GB (AUROC 0.91), and SVM (AUROC 0.80).
• The ANN model demonstrated high sensitivity (0.87) and specificity (0.83) in predicting physician responses.

Conclusions:

• The ANN model significantly outperformed other ML models in predicting individual physician responses to CDSS alerts.
• This study represents the first application of ML models to predict physician alert responses.
• The findings can inform the development of more sophisticated and effective CDSSs for real-world clinical settings.