Assessing entrustability to performance: A longitudinal effort to evolve assessment in advanced pharmacy practice...

The experiential advanced pharmacy practice experience assessment tool needed to be revised and validated to incorporate new guidance from the Accreditation council. One direct patient care assessment was created that included practice-ready questions and entrustable professional activities (EPAs).

A stepwise approach was taken over a five year period to first educate preceptors on a new EPA rating scale, test formatively how the EPA scale performed, and then create a new assessment tool mapping to the new COEPA guidance published in 2022, paying careful attention to domains and questions from the previous APPE evaluations. The new APPE assessment was tested alongside the previous tool and was implemented for the class of 2025. Further analysis was done once the tool was implemented to compare performance and scores generated.

EPAs performed well in the pilot phase, with very few "not observed" scores. Preceptors scoring students on both the previous assessment and the new tool demonstrated high agreement across both evaluations, with an overall agreement of 91.6%. Preceptors showed a 98% agreement to the grades generated by the new tool in the 2024-25 fiscal year.

A new single patient-care APPE assessment tool encompassing practice readiness skills and EPAs that performed accurately and very closely to the previous evaluation was successfully implemented for grading performance on APPEs across different practice settings.