Guidelines for interpretation of some common indicators of residency program performance

Area of Science:

• Medical Education Research
• Internal Medicine Training

Background:

• Assessing residency program quality is crucial for medical education.
• Standardized metrics are needed for reproducible program evaluation.

Purpose of the Study:

• Establish national norms for internal medicine residency program quality indicators.
• Evaluate the reproducibility and precision of these quality metrics.

Main Methods:

• Analyzed data from 364 internal medicine residency programs (1989-1991).
• Included program directors' ratings, certifying exam scores, and pass/fail rates.
• Applied generalizability theory to assess indicator precision across program sizes and years.

Main Results:

• Slight declines observed in knowledge ratings, certification rates, and composite scores over three years.
• Correlations between knowledge ratings and composite scores ranged from 0.44 to 0.60.
• Confidence intervals for performance indicators were large, particularly in smaller programs.

Conclusions:

• Performance indicators lack precision in programs with few residents.
• Program stability across years is noted; aggregating data enhances precision.
• Data can aid in identifying underperforming programs for targeted improvement.