Related Experiment Video
Updated: Feb 18, 2026
Accuracy in Dental Medicine, A New Way to Measure Trueness and Precision
Published on: April 29, 2014
Are Study and Journal Characteristics Reliable Indicators of "Truth" in Imaging Research?
Robert A Frank1, Matthew D F McInnes1, Deborah Levine1
1From the Faculty of Medicine, University of Ottawa, 1053 Carling Ave, Room C120, Ottawa, ON, Canada K1Y 4E9 (R.A.F., T.A.M.); Department of Radiology, University of Ottawa and the Ottawa Hospital, Ottawa, ON, Canada (R.A.F., M.D.F.M., T.A.M.); Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada (M.D.F.M., W.P.); Department of Radiology, Beth Israel Deaconess Medical Center, Harvard University, Boston, Mass (D.L., H.Y.K.); Radiology Editorial Office, Boston, Mass (D.L., H.Y.K.); Department of Radiology, Harvard University, Boston, Mass (J.S.S.); and Clinical Epidemiology and Biostatistics and Bioinformatics, Amsterdam Public Health Research Institute, Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands (P.M.B.).
Abstract:
Purpose To evaluate whether journal-level variables (impact factor, cited half-life, and Standards for Reporting of Diagnostic Accuracy Studies [STARD] endorsement) and study-level variables (citation rate, timing of publication, and order of publication) are associated with the distance between primary study results and summary estimates from meta-analyses. Materials and Methods MEDLINE was searched for meta-analyses of imaging diagnostic accuracy studies, published from January 2005 to April 2016. Data on journal-level and primary-study variables were extracted for each meta-analysis. Primary studies were dichotomized by variable as first versus subsequent publication, publication before versus after STARD introduction, STARD endorsement, or by median split. The mean absolute deviation of primary study estimates from the corresponding summary estimates for sensitivity and specificity was compared between groups. Means and confidence intervals were obtained by using bootstrap resampling; P values were calculated by using a t test. Results Ninety-eight meta-analyses summarizing 1458 primary studies met the inclusion criteria. There was substantial variability, but no significant differences, in deviations from the summary estimate between paired groups (P > .0041 in all comparisons). The largest difference found was in mean deviation for sensitivity, which was observed for publication timing, where studies published first on a topic demonstrated a mean deviation that was 2.5 percentage points smaller than subsequently published studies (P = .005). For journal-level factors, the greatest difference found (1.8 percentage points; P = .088) was in mean deviation for sensitivity in journals with impact factors above the median compared with those below the median. Conclusion Journal- and study-level variables considered important when evaluating diagnostic accuracy information to guide clinical decisions are not systematically associated with distance from the truth; critical appraisal of individual articles is recommended. © RSNA, 2017 Online supplemental material is available for this article.
