A Measurement Science Framework to Optimize CDS for Opioid Use Disorder Treatment in the ED

Area of Science:

• Health Informatics
• Clinical Decision Support Systems
• Opioid Use Disorder Treatment

Background:

• The EMergency department-initiated BuprenorphinE for opioid use Disorder (EMBED) trial found no effect of a clinical decision support (CDS) tool on buprenorphine initiation rates.
• The Agency for Healthcare Research and Quality (AHRQ) developed a CDS Performance Measure Inventory to guide CDS development and evaluation.
• Tailoring AHRQ measures through partner co-design is crucial for evaluating and improving CDS tools like EMBED.

Purpose of the Study:

• To adapt AHRQ inventory measures for evaluating the performance of the EMBED CDS tool.
• To utilize a partner co-design approach to drive improvements in CDS for opioid use disorder treatment.
• To establish a measurement science framework for CDS development and evaluation.

Main Methods:

• Selected and adapted relevant AHRQ inventory measures using a consensus-driven, partner co-design approach.
• Conducted three iterative, multidisciplinary working group sessions with diverse stakeholders.
• Evaluated the developed measures in three emergency departments within a single health system.

Main Results:

• A 25-member partner working group narrowed 13 candidate metrics to 6 priority categories during conceptualization.
• Validated measures included: eligible encounters with CDS engagement (5.0%), teamwork on buprenorphine initiation (39.9%), and proportion of eligible users engaging with EMBED (58.3%).
• Preliminary values for EMBED usage: time spent (29.0 seconds), proportion of orders via EMBED (6.5%), and task completion (13.8%).

Conclusions:

• A measurement science framework, informed by partner co-design, proved feasible for developing measures to guide CDS improvement.
• This co-design approach can be adapted to evaluate other CDS applications.
• Future research should focus on refining and implementing these measures to optimize CDS effectiveness.