Sensitivity of a deep-learning-based breast cancer risk prediction model

Objective.When it comes to the implementation of deep-learning based breast cancer risk (BCR) prediction models in clinical settings, it is important to be aware that these models could be sensitive to various factors, especially those arising from the acquisition process. In this work, we investigated how sensitive the state-of-the-art BCR prediction model is to realistic image alterations that can occur as a result of different positioning during the acquisition process.Approach.5076 mammograms (1269 exams, 650 participants) from the Slovenian and Belgium (University Hospital Leuven) Breast Cancer Screening Programs were collected. The Original MIRAI model was used for 1-5 year BCR estimation. First, BCR was predicted for the original mammograms, which were not changed. Then, a series of different image alteration techniques was performed, such as swapping left and right breasts, removing tissue below the inframammary fold, translations, cropping, rotations, registration and pectoral muscle removal. In addition, a subset of 81 exams, where at least one of the mammograms had to be retaken due to inadequate image quality, served as an approximation of a test-retest experiment. Bland-Altman plots were used to determine prediction bias and 95% limits of agreement (LOA). Additionally, the mean absolute difference in BCR (Mean AD) was calculated. The impact on the overall discrimination performance was evaluated with the AUC.Results.Swapping left and right breasts had no impact on the predicted BCR. The removal of skin tissue below the inframammary fold had minimal impact on the predicted BCR (1-5 year LOA: [-0.02, 0.01]). The model was sensitive to translation, rotation, registration, and cropping, where LOAs of up to ±0.1 were observed. Partial pectoral muscle removal did not have a major impact on predicted BCR, while complete removal of pectoral muscle introduced substantial prediction bias and LOAs (1 year LOA: [-0.07, 0.04], 5 year LOA: [-0.06, 0.03]). The approximation of a real test-retest experiment resulted in LOAs similar to those of simulated image alterations. None of the alterations impacted the overall BCR discrimination performance; the initial 1 year AUC (0.90 [0.88, 0.92]) and 5 year AUC (0.77 [0.75, 0.80]) remained unchanged.Significance.While tested image alterations do not impact overall BCR discrimination performance, substantial changes in predicted 1-5 year BCR can occur on an individual basis.