Validating an EHR report for evaluating a pharmacy-led medication history service

The accuracy of medication histories is critical for patient safety, especially during transitions of care. Although pharmacy-led medication history services have demonstrated clinical value, evaluation of their effectiveness typically relies on manual chart review, which is an unsustainable approach for large-scale quality improvement. This initiative aimed to validate an automated reporting tool embedded in the electronic health record that can be used to evaluate the performance of a pharmacy-led medication history service in the emergency department.

An automated reporting tool was developed using native reporting functionality within the health system's electronic health record. The tool was designed to capture key performance metrics of the service, including the timeliness of medication history review, staff involvement, medication counts, and patient outcomes such as length of stay as well as readmission and revisit rates. Retrospective validation was conducted using a stratified random sample of 121 emergency department admissions between July and December 2024. Data from the automated report were compared to manual chart review results. Most inpatient variables showed 100% agreement, while discrepancies were observed in the number of medications documented (72% accuracy), 30-day readmissions (81%), and 90-day readmissions (68%).

The automated report demonstrated strong accuracy for most metrics, particularly for those related to the hospital admission. Variability in postdischarge data and medication counts highlights the need for refinement of the data extraction logic. Once optimized, the tool offers a scalable solution for continuous performance monitoring and pharmacy service evaluation.