Residency program director evaluations do not correlate with performance on a required 4th-year objective structured...

Area of Science:

• Medical Education Research
• Graduate Medical Education Assessment
• Clinical Skills Evaluation

Background:

• Assessing resident performance is a recognized challenge in medical education.
• Objective measures are sought to complement subjective evaluations.
• Current assessment methods may lack sufficient correlation with objective performance indicators.

Purpose of the Study:

• To determine the correlation between 4th-year Objective Structured Clinical Examination (OSCE) performance and residency program director assessments.
• To investigate the relationship between OSCE scores, class rank, and U.S. Medical Licensing Examination (USMLE) scores.
• To evaluate the alignment of subjective program director evaluations with objective measures of clinical skills and knowledge.

Main Methods:

• Surveyed program directors regarding the performance of 50 medical school graduates.
• Graduates were selected based on high (OSCEHI) and low (OSCELO) performance on a 4th-year OSCE.
• Program directors were blinded to the graduates' specific OSCE scores.

Main Results:

• No significant correlation was found between OSCE scores and program director Likert scale ratings (r < .23, p > .13).
• Program director evaluations showed no significant correlation with class rank or USMLE scores (r < .26, p > .09).
• Objective performance on the OSCE did not align with subjective assessments by program directors.

Conclusions:

• Program director evaluations of resident performance do not reliably correlate with objective assessments of clinical skills or medical knowledge.
• Current subjective evaluation methods may not accurately reflect a resident's objective capabilities.
• Development of more structured and objective assessment tools is recommended for postgraduate training programs.