Malignancy detection in digital mammograms: important reader characteristics and required case numbers

To determine the relationship between heightened levels of reader performance and reader practice in terms of number of cases read and previous experience.

A test set of mammograms was developed comprising 50 cases. These cases consisted of 15 abnormals (biopsy proven) and 35 normals (confirmed at subsequent rescreen). Sixty-nine breast image readers reviewed these cases independently and their performance was measured by recording their individual receiver operating characteristic score (area under the curve), sensitivity, and specificity. These measures of performance were then compared to a range of factors relating to the reader such as years of certification and reporting, number of cases read per year, previous experiences, and satisfaction levels. Correlation analyses using Spearman methods were performed along with the Mann-Whitney test to detect differences in performance between specific reader groups.

Improved reader performance was found for years certified (P = .004), years of experience (P = .0001), and hours reading per week (P = .003) shown by positive statistical significant relationships with Az values (area under receiver operating characteristic curve). Statistical comparisons of Az values scored for individuals who read varying number of cases per year showed that those individuals whose annual mammographic case load was 5000 or more (P = .03) or between 2000 and 4999 (P = .05), had statistically significantly higher scores than those who read less than 1000 cases per year.

The results of this study have shown variations in reader performance relating to parameters of reader practice and experience. Levels of variance are shown and potential acceptance levels for diagnostic efficacy are proposed which may inform policy makers, judicial systems and public debate.