In-training examinations as predictors of resident clinical performance

Area of Science:

• Medical Education
• Residency Training
• Clinical Performance Assessment

Background:

• Predicting resident clinical performance is crucial for patient safety and effective training.
• Previous reliance on cognitive measures alone has proven insufficient for accurate performance prediction.

Purpose of the Study:

• To evaluate the effectiveness of oral in-training examinations (OITEs) compared to written in-training examinations (WITEs) in predicting resident clinical performance.
• To determine if OITEs can accurately identify residents with suboptimal clinical performance.

Main Methods:

• A formative residency evaluation system utilizing yearly OITEs assessing cognitive, psychomotor, and affective domains was employed.
• OITE scores and WITE scores were compared with subsequent faculty ratings of clinical performance.
• Statistical analysis was performed to assess correlations and predictive accuracy, including sensitivity and specificity for identifying "problem" interns.

Main Results:

• No significant correlation was found between WITE scores and clinical performance across all training levels.
• Oral in-training examination scores demonstrated a high correlation with faculty-rated clinical performance.
• OITEs accurately predicted "problem" interns (lower 10% performance) with high significance, sensitivity, and specificity.

Conclusions:

• Oral in-training examinations are a powerful formative evaluation tool for assessing resident competence across multiple domains.
• OITEs provide objective, patient-centered assessments that correlate strongly with clinical performance, unlike WITEs.
• The OITE facilitates early detection and intervention for residents at risk of clinical performance issues, enhancing training outcomes.