Using alert dwell time to filter universal clinical alerts: A machine learning approach.

Area of Science:

• Medical Informatics
• Machine Learning in Healthcare
• Patient Safety

Background:

• Rule-based alerts in computerized physician order entry (CPOE) systems improve patient safety but lack customization.
• This limitation leads to irrelevant alerts and alert fatigue for healthcare providers.

Purpose of the Study:

• To develop and evaluate machine learning models that use alert dwell time and contextual factors to filter irrelevant alerts for physicians.
• To enhance the specificity and reduce alert fatigue in CPOE systems.

Main Methods:

• Utilized five machine learning algorithms with 1,120 features across alert, demographic, environment, diagnosis, prescription, and laboratory categories.
• Employed alert dwell time within a specific time window, optimized via sensitivity analysis, to predict alert relevance.
• Analyzed 813,026 outpatient records from 2020-2021.

Main Results:

• A time window of 0.3-4.0 seconds demonstrated optimal performance, achieving an area under the receiver operating characteristic (AUROC) curve of 0.73 and an area under the precision-recall curve (AUPRC) of 0.97.
• Models incorporating alert and demographic features yielded the best performance (AUROC 0.73).
• Alert and demographic features were the most significant individual contributors to alert relevance prediction.

Conclusions:

• Alert and user/patient demographic features are more critical than clinical features for creating universal context-aware alerts.
• Alert dwell time combined with a time window effectively determines alert trigger status.
• Findings offer insights for developing specific and universal context-aware alerts to improve CPOE system usability and patient safety.