A sequential chart for the audit-based evaluation of screening mammogram interpretation

Auditing has received much attention recently as a method for radiologists to use to evaluate their interpretation of screening mammograms. U.S. Food and Drug Administration regulations require that some sort of audit be in place before a mammography screening facility can receive accreditation. Auditing presents a unique opportunity to monitor accuracy continually and identify problems early. Audit data present unique challenges, however, and appropriate methods must be used to control the risk of errors.

This article introduces a simple method for the task of deciding if a radiologist yields an acceptable positive predictive value based on audit. The method is based on "sequential" decision-making techniques that have found wide application in quality control problems. These techniques are developed for diagnostic radiology and embodied in an easy-to-use decision-making chart.

Several examples, based on audit data from actual mammography facilities, provide insights into the use of these charts and the influence of (a) the selection of standards, (b) the selection of error risks, and (c) radiologist variability. The examples also serve to demonstrate another important property of this method--that is, it specifies the minimum amount of data that has to be collected before any decision can reliably be made.

The chart presented in this article provides a method by which audit data can be used objectively to evaluate the accuracy of screening mammogram interpretation. The method controls the risk of either falsely accepting an unqualified radiologist or falsely rejecting a qualified radiologist. It should be a useful tool to radiologists who must evaluate their own practices.