A deep learning-based multimodal medical imaging model for breast cancer screening

In existing breast cancer prediction research, most models rely solely on a single type of imaging data, which limits their performance. To overcome this limitation, the present study explores breast cancer prediction models based on multimodal medical images (mammography and ultrasound images) and compares them with single-modal models. We collected medical imaging data from 790 patients, including 2,235 mammography images and 1,348 ultrasound images, and conducted a comparison using six deep learning classification models to identify the best model for constructing the multimodal classification model. Performance was evaluated using metrics such as area under the receiver operating characteristic curve (AUC), sensitivity, specificity, precision, and accuracy to compare the multimodal and single-modal classification models. Experimental results demonstrate that the multimodal classification model outperforms single-modal models in terms of specificity (96.41% (95% CI:93.10%-99.72%)), accuracy (93.78% (95% CI:87.67%-99.89%)), precision (83.66% (95% CI:76.27%-91.05%)), and AUC (0.968 (95% CI:0.947-0.989)), while single-modal models excel in sensitivity. Additionally, heatmap visualization was used to further validate the classification performance of the multimodal model. In conclusion, our multimodal classification model shows strong potential in breast cancer screening tasks, effectively assisting physicians in improving screening accuracy.