Design and Content Validation of Three Setting-Specific Assessment Tools for Advanced Pharmacy Practice Experiences

Objective. To develop and validate three assessment tools to measure student performance in clinical settings during required advanced pharmacy practice experiences (APPEs). Methods. Each assessment tool was developed by subject-matter experts and validated using three sources of evidence. Proposed content underwent a job-task analysis by external APPE preceptors, evaluating each outcome in terms of importance and frequency of opportunity for student demonstration. After implementation, tool performance was evaluated using a measure of item applicability and student achievement. Finally, a two-step grade verification process was developed and grade acceptance by preceptor was determined. A priori benchmarks for acceptable tool performance were established for each strategy. Results. The job-task analysis was completed by 131 preceptors (52% response), with items achieving a composite score from 1.08 to11.83 (possible range: 1=most valuable, 25=least valuable). The mean item applicability ranged from 91.3% to 94.1% (n=849 student assessments) during the first year. Preceptors accepted the tool-generated grade in 798 (94%) assessments. Conclusion. Data from three evaluation strategies provide evidence of effective design and use of three unique APPE setting-specific student assessment tools. Pharmacy programs can adopt similar methodologies to develop and validate their assessment tools.