Level of evidence gap in orthopedic research

Level of evidence is the most widely used metric for the quality of a publication, but instances exist in which a Level I study is neither feasible nor desirable. The goal of this study was to evaluate the level of evidence gap in current orthopedic research, which the authors defined as the disparity between the level of evidence that would be required to optimally answer the primary research question and the level of evidence that was actually used. Five orthopedic surgeons (K.D.B., J.B., J.A., S.D.M., W.N.S.) evaluated blinded articles from the first 6 months of 2010 in the Journal of Bone and Joint Surgery (American Volume) (JBJS-Am), classifying the study type and design and extracting a primary research question from each article. Each evaluator then defined the study type and method, along with the level of evidence that would ideally be used to address the primary research question. The level of evidence gap was then calculated by subtracting the actual level of evidence of the manuscript from the level of evidence of the idealized study. Of the 64 JBJS-Am manuscripts eligible for analysis, the average level of evidence was between Level II and III (mean, 2.73). The average level of evidence gap was 1.06 compared with the JBJS-Am-designated level of evidence and 1.28 compared with the evaluators' assessment. Because not all questions require Level I studies, level of evidence alone may not be the best metric for the quality of orthopedic surgery literature. Instead, the authors' concept of a level of evidence gap may be a better tool for assessing the state of orthopedic research publications.