The Predictive Performance of General Surgery Milestones on Postgraduation Outcomes

To establish whether the Accreditation Council for Graduate Medical Education Milestones predict the future performance of general surgery trainees.

Milestones provide bi-annual assessments of trainee progress across 6 competencies. It is unknown whether the Milestones predict surgeon performance after the transition to independent practice.

We performed a retrospective cohort study of surgeons with complete Milestone assessments in the fourth and fifth clinical years who treated patients in acute care hospitals within Florida, New York, and Pennsylvania, 2015-2018. To account for the multiple ways in which the Milestone assessments might predict postgraduation outcomes, we included 120 Milestones features in our elastic net machine learning models. The primary outcome was risk-adjusted patient death or serious morbidity.

A total of 278 general surgeons were included in the study. Milestone assessments 6 months into the fourth clinical year displayed a normal score distribution while multicollinearity and low score discrimination at the final assessment period were detected. Individual Milestones features from the Patient Care, Professionalism, and Systems-based Practice domains were most predictive of patient-related outcomes. For example, surgeons with worse patient outcomes had significantly lower scores in Patient Care 3 when compared with surgeons with better patient outcomes (high DSM, yes: 2.86 vs no: 3.04, P =0.011).

The Milestones features that were most predictive of better patient outcomes related to intraoperative skills, ethical principles, and patient navigation and safety measured 12 to 18 months before graduation. The development of a parsimonious set of evidence-based Milestones that better correlate with surgeon experience could enhance surgical education.