Are Surgical Milestone Assessments Predictive of In-Training Examination Scores?

With the recent utilization of Accreditation Council for Graduate Medical Education developmental milestones as part of resident evaluation, we sought to explore whether milestone-based ratings were associated with American Board of Surgery In-Training Examination (ABSITE) scores.

Mid-year milestone ratings were obtained from the Accreditation Council for Graduate Medical Education Accreditation Database System for years 2014, 2015, and 2016 for all postgraduate years 1-5 general surgery residents in our program and paired with ABSITE scores (n = 69) from January of the following year. Linear regression was used to assess predictor importance of milestones on both ABSITE percentage correct scores and ABSITE percentile scores.

Minimal to small positive correlations were observed between milestones and ABSITE percentile scores (r = 0.09-0.25), while moderately large correlations were observed between milestones and percentage correct scores (r = 0.65-0.76). The Medical Knowledge 1 (MK1) milestone significantly predicted ABSITE percentage correct scores, and explained 60% of the variance (adjusted R² = 0.603). MK1 also significantly predicted ABSITE percentile scores, although weaker in magnitude, with MK1 explaining 20% of the variance (adjusted R² = 0.197). Postgraduate year level and other milestones were not influential predictors of ABSITE scores.

The mid-year MK1 milestone rating was predictive of ABSITE scores and may serve as a useful marker for Clinical Competency Committees to identify residents who could benefit from additional support to prepare for the ABSITE, although given the small exploratory nature of this study, additional research is still needed.