Ranking: a year three follow-up in a different institution

Area of Science:

• Medical Education
• Residency Training
• Performance Evaluation

Background:

• Semiobjective evaluation of resident performance using faculty ranking was introduced in 1985.
• The predictive validity of pooled faculty rankings has been previously emphasized.
• Previous studies noted correlations between resident examination scores and faculty rankings.

Purpose of the Study:

• To evaluate the continued predictive nature of pooled faculty rankings for resident performance.
• To assess the consistency of faculty consensus in resident evaluation.
• To examine the utility of ranking exceptions in resident counseling.

Main Methods:

• Retrospective analysis of resident performance data and faculty rankings.
• Statistical correlation of faculty rankings with in-training and board examination scores.
• Observation of ranking consistency over time and across different faculty group sizes.

Main Results:

• Pooled faculty rankings consistently predict resident performance and correlate with examination scores.
• Faculty rankings demonstrated predictive accuracy for subsequent rankings.
• A minimum of five faculty provided a cohesive consensus predictive of larger groups.
• Ranking stratification exceptions (e.g., junior residents outranking seniors) proved highly informative for counseling.
• Rankings showed sensitivity to changes in performance for residents under probation.

Conclusions:

• Pooled faculty ranking remains a valuable and predictive tool for semiobjective resident performance evaluation.
• Faculty consensus in ranking is cohesive and reliable, even with small groups.
• Analyzing deviations from expected performance stratification is crucial for effective resident counseling and support.