The Validation of a Case-Based, Cumulative Assessment and Progressions Examination

Area of Science:

• Pharmacy education
• Medical assessment

Background:

• Developing high-stakes examinations for student progression is crucial in pharmacy education.
• Internal assessments require rigorous validation to ensure fairness and accuracy.

Purpose of the Study:

• To evaluate the content validity, criterion validity, and reliability of the internally developed, case-based, cumulative, high-stakes third-year Annual Student Assessment and Progression Examination (P3 ASAP Exam).

Main Methods:

• Content validity was established through a comprehensive writing and reviewing process.
• Criterion validity was assessed by correlating P3 ASAP Exam scores with the Pharmacy Curriculum Outcomes Assessment (PCOA).
• Reliability was determined using psychometric analysis, specifically the Kuder-Richardson coefficient (KR-20), over four years of student performance data.

Main Results:

• The P3 ASAP Exam demonstrated content validity by effectively representing didactic coursework and professional outcomes.
• A strong correlation between P3 ASAP Exam and PCOA scores confirmed criterion validity.
• Consistent student performance trends since 2012, indicated by KR-20, confirmed the examination's reliability.

Conclusions:

• Internally developed, high-stakes, cumulative progression examinations can be both valid and reliable.
• A robust writing-review process and thorough psychometric analysis are key to implementing successful internal assessments in pharmacy schools.