Photo coding: a pragmatic approach to measuring intervention fidelity in a pharmacy system re-design

Measuring intervention fidelity can help justify the transition of evidence-based practices into everyday community-based practices by helping identify components of the intervention supporting the intervention's uptake. Current methods to measure fidelity include self-reported data, observational data using audio or video recordings, and in-vivo observations, but these are limited due to cost-ineffectiveness, time inefficiency, and external validity threats. Photo coding is proposed as a new, pragmatic method of observational data to measure intervention fidelity, as was used in a pharmacy system re-design intervention (Senior Safe™) to improve over-the-counter (OTC) medication safety for older adults.

Guided by human factors engineering principles, Senior Safe re-shelved OTC medications based on safety level, and signage was added to designate safer and high-risk products. Senior Safe was implemented by pharmacy leadership and maintained by pharmacy staff. Pharmacy leadership and researchers collaborated to take photos of Senior Safe two times (Time 1 and Time 2), at least three months apart, to examine intervention layout and medication categorization fidelity. A Layout Codebook was constructed to evaluate the conformity of signage to ergonomic principles. A Medication Categorization Codebook was designed to assess whether the signage was properly allocated to the designated products. Two research assistants and two PharmD students coded the photos. A fidelity standard of ≥ 80% concordance with intervention guidelines was used to signify high-fidelity.

Fidelity was assessed within 67 pharmacy sites implementing Senior Safe. All sites achieved ≥ 80% fidelity concordance between Time 1 and Time 2 for the Layout Codebook. Alternatively, for the Medication Categorization Codebook, 97% of sites met high-fidelity standards during Time 1, compared to 85% of sites at Time 2, indicating statistically significant negative drift. Overall, over half of all Time 1 and Time 2 concordance rates involving the Layout and Medication Categorization Codebooks exceeded the ≥ 80% high-fidelity standard.

Measuring intervention fidelity is useful in determining whether interventions are sustained and successfully transitioned into community-based practice. Photo coding is an innovative approach to measure intervention fidelity and allows researchers to identify multiple layers of concordance and discordance to intervention guidelines. Organizations are encouraged to investigate intervention fidelity, including identifying discordance and adaptation needs.