The poor quality of early evaluations of magnetic resonance imaging

To study the quality of early research on the clinical efficacy of diagnostic imaging with magnetic resonance, we assessed 54 evaluations published in the first four years after introduction of this modality using ten commonly accepted criteria of research methodology. The terms sensitivity, specificity, false-positive or false-negative, accuracy, and predictive values were used infrequently. Nineteen percent of the evaluations used three terms appropriately, 48% used one or two terms, and 33% used none. Data were presented appropriately for one or more of the five terms in 59% of evaluations. A "gold standard" comparison with the results of an independent procedure, such as surgical or autopsy findings, was presented in 22% of evaluations. Results of another imaging procedure were described in 63% of evaluations. Only one evaluation clearly described a prospective study design, although 11 evaluations apparently were planned in advance. Not one evaluation contained an appropriate statistical analysis of the distributions of quantitative readings, "blinded" image readers to diagnosis or other test results, measured observer error, or randomized the order of magnetic resonance imaging and other imaging procedures. We conclude that health care professionals paying for expensive innovative diagnostic technology should demand better research on diagnostic efficacy.