Approaching Impact Meaningfully in Medical Education Research

Medical education research faces increasing pressure to demonstrate impact and utility. These pressures arise amidst a climate of accountability and within a culture of outcome measurement. Conventional metrics for assessing research impact such as citation analysis have been adopted in medical education, despite researchers' assertion that these quantitative measures insufficiently reflect the value of their work. Every knowledge community has its own definitions of what counts as knowledge, how that knowledge should be produced, and how the quality of that knowledge production should be evaluated. Definitions of impact and knowledge shape and constrain researchers' foci and endeavors. Therefore, metrics that meaningfully evaluate the knowledge outputs of researchers need to be defined within each field. It is time for medical education research, as a field, to examine how to measure research impact and carefully consider the broader implications these measures may have. The authors discuss developments in research metrics more broadly, then critically examine impact metrics currently used in the medical education field and propose alternatives to more meaningfully track and represent impact in medical education research. Grey metrics and narrative impact stories to more fully capture the richness and nuanced nature of impact in medical education research are introduced. The authors advocate for a continual examination of how impact is defined, eschewing unquestioned use of conventional metrics. A new conversation is needed, as well as a research agenda to help medical education conceptualize and study metrics more appropriate for the field.