Development, Validation, and Reliability of a P1 Objective Structured Clinical Examination Assessing the National...

Area of Science:

• Pharmacy Education
• Clinical Skills Assessment
• Entrustable Professional Activities

Background:

• The Objective Structured Clinical Examination (OSCE) is crucial for evaluating pharmacy students' clinical readiness.
• National entrustable professional activities (EPAs) provide a framework for assessing competency in pharmacy practice.
• Aligning OSCEs with EPAs ensures relevance to advanced pharmacy practice experiences.

Purpose of the Study:

• To evaluate the performance of first-year pharmacy students on a revised OSCE.
• To identify risk factors associated with poor performance on the OSCE.
• To assess the validity and reliability of the revised OSCE.

Main Methods:

• Developed an OSCE based on national EPAs and cross-mapped to Accreditation Council for Pharmacy Education outcomes.
• Investigated risk factors using baseline characteristics and academic performance data.
• Assessed reliability through blinded re-grading and Cohen's kappa analysis.

Main Results:

• 50.8% of 65 students passed all OSCE stations on the first attempt.
• Students who passed on the first attempt had higher Health Sciences Reasoning Test scores and GPA.
• No statistically significant predictors of poor performance were identified in logistic regression.
• Cohen's kappa values indicated moderate to substantial reliability (0.404-0.708).

Conclusions:

• The revised OSCE demonstrates good validity and reliability for assessing pharmacy students.
• Predictors of poor performance could not be definitively identified in this cohort.
• The OSCE effectively measures student progress towards L1 entrustment on national EPAs.