Monitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity...

Area of Science:

• Health Informatics
• Clinical Decision Support Systems
• Quality Improvement

Background:

• Clinical decision support (CDS) tools in electronic health records (EHRs) are key for quality improvement but require effective monitoring.
• Current monitoring methods (self-reports, observation) are data-intensive and prone to bias.
• There is a need for objective, scalable methods to assess CDS tool impact.

Purpose of the Study:

• To develop and demonstrate a novel monitoring method using EHR activity data.
• To evaluate CDS tools within a National Cancer Institute-sponsored tobacco cessation program (Cancer Center Cessation Initiative - C3I).
• To identify areas for improving CDS alert design and adoption.

Main Methods:

• Developed EHR-based metrics to monitor two CDS tools: a smoking assessment screening alert and a treatment support alert.
• Measured alert completion rates and burden (frequency, time spent) using EHR activity data over 12 months.
• Compared metrics across seven cancer clinics to identify implementation variations and improvement opportunities.

Main Results:

• Screening alert completion rates varied by clinic (0.32-0.55) but remained stable over time.
• Support alert was acted upon in 87.3% of encounters, leading to quit-readiness identification (12%) and referrals (2%).
• Alert burden analysis showed alerts fired multiple times before completion, with significant time spent handling them, informing design improvements.

Conclusions:

• EHR activity metrics provide a nuanced understanding of CDS alert success and burden.
• These metrics are valuable for guiding the adaptation and optimization of CDS tool implementation.
• The developed monitoring approach is scalable and applicable across various clinical settings.