Do OSCE progress test scores predict performance in a national high-stakes examination?

Area of Science:

• Medical Education
• Assessment and Evaluation
• Competency-Based Education

Background:

• Progress testing, involving repeated assessments with feedback, aligns well with competency-based frameworks requiring frequent formative evaluations.
• The Objective Structured Clinical Examination (OSCE) progress test assesses clinical skill progression in medical trainees.
• This study aimed to validate the OSCE progress test by correlating its scores with a national high-stakes examination.

Purpose of the Study:

• To establish evidence for the utility of the OSCE progress test in medical training.
• To demonstrate a significant association between OSCE progress test scores and performance on a national high-stakes examination.

Main Methods:

• Compared 8 years of Internal Medicine Residency OSCE (IM-OSCE) progress test data with the Royal College of Physicians and Surgeons of Canada Comprehensive Objective Examination in Internal Medicine (RCPSC IM examination) scores (n=180).
• Calculated correlations between IM-OSCE and RCPSC IM examination scores (written and performance-based components).
• Utilized logistic regression to assess if IM-OSCE scores predict an elevated risk of failure on the RCPSC IM examination.

Main Results:

• Significant correlations were found between IM-OSCE scores and the performance-based component of the RCPSC IM examination (0.316 to 0.554).
• Significant correlations were also observed between IM-OSCE scores and the written component of the RCPSC IM examination (0.305 to 0.516).
• Logistic regression indicated that PGY-2 and PGY-4 IM-OSCE scores predicted an elevated risk of failure on both components of the national examination.

Conclusions:

• The study provides robust evidence supporting the use of OSCE progress testing in medical education.
• A clear correlation exists between OSCE progress test scores and performance on a national high-stakes examination.
• OSCE progress test scores are demonstrably predictive of future success on high-stakes medical assessments.