Leveraging an Implementation Fidelity Framework to Assess Quality in Experiential Education

Objective. Fidelity metrics can provide insight into the extent to which experiential programs are implemented as they were designed to be. The UNC Eshelman School of Pharmacy at the University of North Carolina at Chapel Hill implemented a new curriculum that included a series of three, two-month introductory pharmacy practice experiences (IPPEs). The objective of this study was to design a logic model for the first IPPE within the series, identify key variables to program implementation, define fidelity indices and benchmarks, and compute a single fidelity score for each IPPE site.Methods. Data were collected from the course syllabus, learning outcomes, assignments, and evaluations from students and preceptors for 50 sites that had hosted 147 students for IPPEs. A logic model was defined to describe inputs, activities, outputs, and outcomes of the IPPE. Data were reviewed for key variables and measures to include in the fidelity framework and then a fidelity score was generated for each site.Results. Twenty-four variables were identified across three components that were deemed critical for experience implementation (eg, patient care activities, preceptor compliance, and overall site training and evaluation). The mean fidelity score for all sites was 59.1% (SD 16.4%).Conclusion. A logic model and fidelity framework provided an objective method to assess the extent to which practice sites delivered the IPPE course. This work could be used by schools as a basis for individualizing quality assurance efforts.