A deep learning-based multimodal medical imaging model for breast cancer screening

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Current breast cancer prediction models often rely on single imaging modalities, limiting diagnostic performance.
• Integrating multimodal data presents an opportunity to enhance prediction accuracy.

Purpose of the Study:

• To develop and evaluate breast cancer prediction models using multimodal (mammography and ultrasound) versus single-modal imaging data.
• To compare the performance of six deep learning classification models for multimodal breast cancer prediction.
• To identify the optimal deep learning model for multimodal breast cancer classification.

Main Methods:

• Collected and analyzed 2,235 mammography and 1,348 ultrasound images from 790 patients.
• Developed and compared multimodal and single-modal deep learning classification models.
• Evaluated model performance using metrics including AUC, sensitivity, specificity, precision, and accuracy.

Main Results:

• The multimodal classification model achieved higher specificity (96.41%), accuracy (93.78%), precision (83.66%), and AUC (0.968) compared to single-modal models.
• Single-modal models showed higher sensitivity.
• Heatmap visualization confirmed the multimodal model's classification performance.

Conclusions:

• Multimodal breast cancer classification models integrating mammography and ultrasound data show significant potential for improving screening accuracy.
• The developed multimodal model can effectively assist physicians in enhancing breast cancer screening.
• Further research can explore integrating additional data types to further optimize prediction models.