Clinical decision support alert malfunctions: analysis and empirically derived taxonomy

Area of Science:

• Health Informatics
• Clinical Decision Support Systems
• Medical Informatics

Background:

• Clinical decision support (CDS) systems are crucial for patient safety and quality of care.
• Malfunctions in CDS alerts can lead to significant patient safety risks and workflow disruptions.
• Understanding the patterns and causes of CDS alert malfunctions is essential for system improvement.

Purpose of the Study:

• To develop an empirically derived taxonomy of clinical decision support (CDS) alert malfunctions.
• To identify and categorize the common causes, discovery methods, timing, and impact of CDS alert malfunctions.
• To provide a framework for anticipating, preventing, and resolving CDS alert issues.

Main Methods:

• Utilized a mixed-methods approach including site visits, interviews, surveys, and analysis of CDS firing rates and overrides.
• Conducted a multi-round, manual, iterative card sort with 68 CDS alert malfunction cases from 14 diverse US sites.
• Developed a multi-axial taxonomy based on qualitative and quantitative data analysis.

Main Results:

• Identified four primary axes for classifying CDS alert malfunctions: cause, discovery, onset, and effect on rule firing.
• Frequent causes included build errors, conceptualization errors, and new concept introductions.
• Malfunctions commonly resulted in false positives (unnecessary alerts) and false negatives (missed alerts).

Conclusions:

• CDS alert malfunctions are frequent and recurring across different healthcare organizations and electronic health record systems.
• The developed taxonomy formalizes common issues, aiding CDS developers in preventing and resolving malfunctions.
• Proactive identification and resolution of these malfunctions are critical for optimizing CDS effectiveness and patient safety.