Multi-Scale Vision Transformer with Optimized Feature Fusion for Mammographic Breast Cancer Classification

Background: Breast cancer remains one of the leading causes of mortality among women worldwide, highlighting the critical need for accurate and efficient diagnostic methods. Methods: Traditional deep learning models often struggle with feature redundancy, suboptimal feature fusion, and inefficient selection of discriminative features, leading to limitations in classification performance. To address these challenges, we propose a new deep learning framework that leverages MAX-ViT for multi-scale feature extraction, ensuring robust and hierarchical representation learning. A gated attention fusion module (GAFM) is introduced to dynamically integrate the extracted features, enhancing the discriminative power of the fused representation. Additionally, we employ Harris Hawks optimization (HHO) for feature selection, reducing redundancy and improving classification efficiency. Finally, XGBoost is utilized for classification, taking advantage of its strong generalization capabilities. Results: We evaluate our model on the King Abdulaziz University Mammogram Dataset, categorized based on BI-RADS classifications. Experimental results demonstrate the effectiveness of our approach, achieving 98.2% for accuracy, 98.0% for precision, 98.1% for recall, 98.0% for F1-score, 98.9% for the area under the curve (AUC), and 95% for the Matthews correlation coefficient (MCC), outperforming existing state-of-the-art models. Conclusions: These results validate the robustness of our fusion-based framework in improving breast cancer diagnosis and classification.