Determining efficacy of mammographic CAD systems

Computer-aided detection (CAD) system sensitivity estimates without a radiologist in the loop are straightforward to measure but are extremely data dependent. The only relevant performance metric is improvement in CAD-assisted radiologist sensitivity. Unfortunately, this is difficult to accurately assess. Without a large study measuring the improvement in CAD-assisted radiologist sensitivity over the same cases, it is not possible to make valid comparisons between systems. As multiple CAD systems become commercially available, comparison issues need to be explored and resolved. Data from clinical trials of 2 systems are examined. Statistical hypothesis tests are applied to these data. Additionally, sensitivities of 2 systems are compared from an experiment testing over the same 120 cases. Even with large databases, there is not sufficient evidence to conclude performance differences exist between the 2 systems. It is prohibitively expensive to show conclusive sensitivity differences between commercially available mammographic CAD systems.