Investigating a Model to Predict Milestones From Entrustable Professional Activity Levels for Medicine-Pediatrics...

Residency programs will need to assess residents using both Accreditation Council for Graduate Medical Education (ACGME) milestones and entrustable professional activities (EPAs) to meet ACGME and American Board of Pediatrics requirements in the coming year. Identifying ways to optimize assessment efforts using both frameworks is important. The authors applied a model developed for predicting milestone levels from EPA entrustment-supervision levels among categorical pediatrics residents to make predictions for internal medicine-pediatrics residents.

During three academic years (2021-2024), the authors conducted a multi-site prospective cohort study at 8 United States internal medicine-pediatrics residency programs. They generated predictions of the 22 ACGME pediatrics milestones from the 17 general pediatrics EPAs determined by program clinical competency committees (CCCs). Predicted milestones were compared with actual ACGME-reported milestones. Overall association between predicted and reported milestones was estimated using a partial correlation coefficient (adjusting for program, resident, and competency) and fitted mixed effects regressions for differences between predicted and reported milestone levels with competency as a fixed effect and program and resident as random effects.

Across 378 internal medicine-pediatrics residents, 6101 EPA entrustment-supervision levels and 7784 ACGME milestone levels were collected. Across all competencies, the marginal mean probability of an exact match between CCC reported and model predicted milestone levels was 38%; however, the likelihood of being within 0.5 level was 93% and within 1 level was over 99%.

The authors present a method for predicting milestone levels from EPA levels. Predicted milestone levels should be vetted with CCCs.