SwinCAMF-Net: Explainable Cross-Attention Multimodal Swin Network for Mammogram Analysis

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Oncology

Background:

• Breast cancer remains a significant cause of mortality in women, underscoring the need for earlier and more accurate diagnostic methods.
• Current mammography systems often lack integration of volumetric and clinical data, limiting diagnostic accuracy.
• Existing deep learning models typically use 2D features and lack cross-modal reasoning, hindering comprehensive analysis.

Purpose of the Study:

• To develop SwinCAMF-Net, a multimodal deep learning network utilizing cross-attention for improved joint classification and segmentation of breast lesions.
• To integrate multi-view mammography, 3D region of interest (ROI) volumes, and clinical metadata for enhanced diagnostic capabilities.

Main Methods:

• SwinCAMF-Net employs a Swin transformer for mammographic feature extraction and a 3D CNN for volumetric data analysis.
• A clinical projection module incorporates patient metadata, and a cross-attentive fusion (CAF) module aligns multimodal features.
• The fused features are used for both malignancy classification and lesion segmentation.

Main Results:

• SwinCAMF-Net achieved high performance metrics: 0.978 accuracy, 0.998 AUC-ROC, and 0.944 F1-score for classification.
• Segmentation performance reached a Dice coefficient of 0.931.
• Ablation studies showed the CAF module improved performance by up to 6.9%, validating its effectiveness in multimodal fusion.

Conclusions:

• SwinCAMF-Net effectively integrates multimodal data using cross-attention for advanced breast cancer analysis.
• The model offers improved diagnostic performance and clinical interpretability, showing potential for AI-assisted screening and radiology decision support.