Finding the optimal recall rate in breast cancer screening: results from the ROCS study

In breast cancer screening, determining the optimal balance between the number of screen-detected cancer cases and false-positive recalls is essential. This study explored the relationship between these indicators for the Dutch Digital Mammography Screening Program and aimed to determine the optimal recall rate.

From March to June 2019, 21 screening radiologists provided continuous Probability-of-Malignancy (PoM) scores during their standard reading sessions. Scores ranged from 'no suspicion' (PoM = -100) to 'highest suspicion' (PoM = 100). Follow-up data were obtained in June 2024 and included recall decisions after double reading, outcomes of further assessments (false positive or screen-detected cancer), and interval cancer diagnoses. Recall-detection and receiver operating characteristic (ROC) curves were constructed for each reader and averaged to obtain summary curves, with error bars computed from hierarchical bootstrapping of cases within readers (1000 resamples). The overall screening performance was quantified using the area under the ROC curve (AUC).

The final dataset comprised 40,829 screening records with 315 cancer cases. The overall recall rate was 2.33%, and the cancer detection rate after double reading was 6.4 per 1000 screens. Radiologist performance was high (AUC = 0.91). Moving the operating point results in either substantially lower cancer detection or relatively little gain.

This prospective study identified the trade-off between unconditional recall and detection rates, as well as conditional sensitivity and specificity. We found that Dutch screening radiologists perform at a high level and operate at a point that seems to optimize the false-positive recall and cancer detection rate trade-off.

Question Breast cancer screening requires a good balance between detection and false-positive rate. However, the interrelationship between these rates, and thus the optimal recall, is unknown. Findings Overall, the Dutch screening radiologists performed with high accuracy, and the current operating point optimizes the trade-off between cancer detection and false-positive recalls. Clinical relevance ROCS provides an efficient method to evaluate performance and determine target values based on data from screening practice. This method applies to various screening programs. Screening evaluation is essential for achieving and maintaining a positive benefit-to-harm ratio for participants.