Impact of computer-aided detection prompts on the sensitivity and specificity of screening mammography

To determine the value of computer-aided detection (CAD) for breast cancer screening.

Two sets of mammograms with known outcomes were used in two studies. Participants in both studies read the films with and without the benefit of a computer aid. In both studies, the order of reading sessions was randomised separately for each reader. The first set of 180 films, used in study 1, included 20 false-negative interval cancers and 40 screen-detected cancers. The second set of 120 films, used in study 2, was designed to be favourable to CAD: all 44 cancer cases had previously been missed by a film reader and cancers prompted by CAD were preferentially included.

The studies were conducted at five UK screening centres between January 2001 and April 2003.

Thirty radiologists, five breast clinicians and 15 radiographers participated.

All cases in the trial were digitised and analysed using the R2 ImageChecker version 2.2. Participants all received training on the use of CAD. In the intervention condition, participants interpreted cases with a prompt sheet on which regions of potential abnormality were indicated.

The sensitivity and specificity of participants were measured in both intervention and control conditions.

No significant difference was found for readers' sensitivity or specificity between the prompted and unprompted conditions in study 1 [95% confidence index (CI) for sensitivity with and without CAD is 0.76 to 0.80, for specificity it is 0.81 to 0.86 without CAD and 0.81 to 0.87 with CAD]. No statistically significant difference was found between the sensitivity and specificity of different groups of film reader (95% CI for unprompted sensitivity of radiologists was 0.75 to 0.81, for radiographers it was 0.71 to 0.81, prompted sensitivity was 0.76 to 0.81 for radiologists and 0.69 to 0.79 for radiographers). Thirty-five readers participated in study 2. Sensitivity was improved in the prompted condition (0.81 from 0.78) but the difference was slightly below the threshold for statistical significance (95% CI for the difference -0.003 to 0.064). Specificity also improved (0.87 from 0.86); again, the difference was not significant at 0.05 (95% CI -0.003 to 0.034). A cost-effectiveness analysis showed that computer prompting increases cost.

No significant improvement in film readers' sensitivity or specificity or gain in cost-effectiveness was established in either study. This may be due to the system's low specificity, its relatively poor sensitivity for subtle cancers or the fact the prompts cannot serve as aids to decision-making. Readers may have been better able to make use of the prompts after becoming more accustomed to working with them. Prompts may have an impact in routine use that is not detectable in an experimental setting. Although the case for CAD as an element of the NHS Breast Screening Programme is not made here, further research is required. Evaluations of new CAD tools in routine use are underway and their results should be given careful attention.