Qualitative Study of Orthopedic Surgery Milestones 1.0: Burdens and Benefits

The purpose of this study was to explore the context and mechanisms by which the first set of Milestones impacted the processes of the Clinical Competency Committee, how programs have incorporated the Milestones into their program, and to understand more about the resident perspective in order to improve Orthopedic Surgery Milestones.

In 2020 all 201 accredited orthopedic surgery residency programs were invited to participate in qualitative telephone interviews to assess their experience with the Milestones and complete a supplemental online survey about their Clinical Competency Committee. Participants were comprised of a self-selected sample and complemented by a purposeful sample to ensure a breadth of perspectives. Interview data were analyzed using template analysis. Survey data were analyzed with descriptive statistics.

Interviews were completed with 101 individuals from 47 programs (23% of all programs). The two overarching themes were implementation and impact of Milestones. Subthemes within implementation were substantial variability in approaches to Milestone use in curriculum and assessment, faculty development, and methods to introduce residents to the Milestones assessment framework. The large number of subcompetencies created a significant burden for almost all programs. The structure of the Milestones was also viewed as poorly aligned with the variable design of rotation schedules across programs. Milestones have the potential to offer valuable feedback for trainees and programs overall.

While some benefits were noted and most programs appreciated the intent, the structure and design of the initial set of orthopedic surgery Milestones created substantial challenges for these programs. The results of this study helped guide a revision of the Orthopedic Surgery Milestones. These results can also be used by program leadership to encourage reflection around past, current, and future utilization of the Milestones framework. Further research will be needed to determine the impact of the revision on programs.