Scoring objective structured clinical examinations using video monitors or video recordings

Area of Science:

• Medical Education
• Pharmacy Education

Background:

• Objective Structured Clinical Examinations (OSCEs) are crucial for assessing clinical skills in pharmacy students.
• Standardized scoring methods are essential for reliable and valid OSCE evaluations.

Purpose of the Study:

• To compare the outcomes of scoring OSCEs using real-time video monitoring versus reviewing recorded videotapes.
• To determine if these two scoring methods yield comparable results for pharmacy students.

Main Methods:

• Second- and third-year Doctor of Pharmacy (PharmD) students participated in 3-station OSCEs.
• Sixty randomly selected encounters were scored by faculty observers in real-time via video monitors.
• The same encounters were rescored by the same investigators one month later using recorded videotapes.

Main Results:

• Excellent intra-rater reliability was observed between real-time and videotaped scoring (ICC 3,1 of 0.951 for P2, 0.868 for P3).
• However, significant discrepancies in pass/fail determinations occurred: 13.3% of students shifted from pass (real-time) to fail (video), and 3.3% shifted from fail (real-time) to pass (video).

Conclusions:

• While overall reliability is high, real-time and videotaped OSCE scoring methods produce different pass/fail determinations.
• These distinct outcomes indicate that real-time observation and video recording are not interchangeable methods for scoring OSCEs in pharmacy education.