Association of ACGME Milestones With Other Performance Measures in General Surgery: A Meta-Analytic Study

Area of Science:

• Medical Education
• Surgical Training
• Assessment and Evaluation

Background:

• Accreditation Council for Graduate Medical Education (ACGME) Milestone ratings are used for general surgery trainee assessment.
• Validity evidence, including correlations with learning and early-career outcomes, is crucial for using Milestones as formative feedback.
• This meta-analysis investigates the association between Milestone ratings and various performance measures.

Purpose of the Study:

• To estimate the effect size of the association between general surgery Milestone ratings and other performance measures.
• To determine the validity of Milestone ratings as predictors of trainee performance and future outcomes.

Main Methods:

• A systematic review and meta-analysis of 16 studies were conducted.
• Data were extracted from electronic databases and reference lists, yielding 445 effect sizes.
• A 3-level meta-analysis accounted for clustering and dependency, with moderated analyses exploring heterogeneity.

Main Results:

• Milestone ratings significantly correlated with Entrustable Professional Activities (r = 0.59) and the American Board of Surgery In-Training Examination (r = 0.39).
• No significant correlations were found with United States Medical Licensing Examination scores, social-emotional outcomes, patient outcomes, or residency application factors.
• Training level, Milestones competency category, and Milestones version did not moderate these effect sizes.

Conclusions:

• ACGME Milestone ratings in general surgery demonstrate strong correlations with specific performance indicators like EPAs and ABSITE.
• The validity of Milestones for predicting broader outcomes, including patient outcomes and licensing exams, requires further investigation.
• Milestone ratings may serve as valuable formative feedback tools when correlated with relevant performance metrics.