Transitioning Competency-Based Communication Assessments to the Online Platform: Examples and Student Outcomes

Area of Science:

• Pharmacy Education
• Medical Simulation
• Health Professions Education

Background:

• The COVID-19 pandemic necessitated a shift from in-person to virtual learning environments in pharmacy education.
• Objective Structured Clinical Examinations (OSCEs) are crucial for assessing communication skills, but virtual adaptations lack comparative outcome data.
• Guidance on virtual communication-based OSCEs remains limited.

Purpose of the Study:

• To describe methods for conducting virtual communication-based OSCEs in pharmacy education.
• To compare student performance in virtual versus in-person OSCEs.
• To provide insights into adapting OSCEs for remote assessment.

Main Methods:

• The University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences conducted two virtual communication-based OSCEs (patient counseling, medical history gathering) in 2020.
• Student performance data from virtual OSCEs were compared to 2019 in-person OSCE data.
• Mann Whitney U Tests were used for statistical comparison of student outcomes.

Main Results:

• The 2020 cohort (virtual OSCEs) scored significantly higher than the 2019 cohort (in-person OSCEs) across all assessed variables, including communication, empathy, and professionalism.
• Higher scores were observed in effective verbal/non-verbal communication, patient-friendly education, encounter organization, empathy, trust-building, and professionalism.
• Overall scores were also significantly higher in the virtual OSCE group.

Conclusions:

• While virtual OSCEs demonstrated higher student performance, the small effect sizes suggest minimal practical differences.
• The observed score inflation may be influenced by pandemic-related adjustments in grading patterns.
• The study provides a framework for virtual OSCE implementation but highlights the need for further research on assessment validity and grading consistency.