Mammographic system performance using an image reading qualification method

Our goal was to evaluate mammography systems based on microcalcifications and fiber detection using a statistical phantom (ALVIM, model TRM 18-209, Nuclear Associates) image readings. ALVIM phantom images were acquired under diverse exposure conditions with various equipments, and 5 radiologists with similar expertise reported their findings. The reading performance in the detection of microcalcifications and fibers of different sizes was measured by simulation of equivalent breast tissue with 4.5 and 6.5 cm thicknesses. We determined kappa values, ROC curves, and kappa probability density and detection rates with dedicated software developed locally. The statistical results generated three kappa (K) ranges that allowed quantification of the detection performance at three quality levels: unacceptable (K ≤ 0.64), achievable (K ≥ 0.70) and acceptable (0.64 < K < 0.70). An extensive database permitted a comparison of the reading performance with 99.5% reliability (p < 0.005). The comparison showed a larger dispersion of the kappa values for the images with low contrast generated with mammography equipment which was not properly calibrated, showing that the method is able to detect the performance changes associated with the loss of image quality.