Supplemental Milestones for Emergency Medicine Residency Programs: A Validation Study

Emergency medicine (EM) residency programs may be 36 or 48 months in length. The Residency Review Committee for EM requires that 48-month programs provide educational justification for the additional 12 months. We developed additional milestones that EM training programs might use to assess outcomes in domains that meet this accreditation requirement. This study aims to assess for content validity of these supplemental milestones using a similar methodology to that of the original EM Milestones validation study.

A panel of EM program directors (PD) and content experts at two institutions identified domains of additional training not covered by the existing EM Milestones. This led to the development of six novel subcompetencies: "Operations and Administration," "Critical Care," "Leadership and Management," "Research," "Teaching and Learning," and "Career Development." Subject-matter experts at other 48-month EM residency programs refined the milestones for these subcompetencies. PDs of all 48-month EM programs were then asked to order the proposed milestones using the Dreyfus model of skill acquisition for each subcompetency. Data analysis mirrored that used in the original EM Milestones validation study, leading to the final version of our supplemental milestones.

Twenty of 33 subjects (58.8%) completed the study. No subcompetency or individual milestone met deletion criteria. Of the 97 proposed milestones, 67 (69.1%) required no further editing and remained at the same level as proposed by the study authors. Thirty milestones underwent level changes: 15 (15.5%) were moved one level up and 13 (13.4%) were moved one level down. One milestone (1.0%) in "Leadership and Management" was moved two levels up, and one milestone in "Operations and Administration" was moved two levels down. One milestone in "Research" was ranked by the survey respondents at one level higher than that proposed by the authors; however, this milestone was kept at its original level assignment.

Six additional subcompetencies were generated and assessed for content validity using the same methodology as was used to validate the current EM Milestones. These optional milestones may serve as an additional set of assessment tools that will allow EM residency programs to report these additional educational outcomes using a familiar milestone rubric.