A Hybrid CNN-Transformer Deep Learning Model for Differentiating Benign and Malignant Breast Tumors Using Multi-View...

IntroductionConventional single-image analysis often fails to capture the comprehensive features of breast lesions, limiting the diagnostic accuracy of ultrasound. This study introduces a computer-aided diagnosis (CAD) system that emulates the clinical multi-view diagnostic process to improve reliability.MethodsA novel hybrid deep learning model was proposed for breast cancer diagnosis task using multi-view ultrasound images, which integrates a Convolutional Neural Network (CNN) module with a Transformer-based module. The pre-trained EfficientNetV2 was used to extract spatial features, and the Transformer-based encoder was applied to fuse information from multiple, unordered ultrasound images of a single lesion. This retrospective study utilized a primary internal dataset from 840 patients and an independent external dataset from 133 patients. A strict lesion-level data partitioning strategy was implemented to prevent data leakage. The performance of the proposed model was evaluated against a baseline single-image model, advanced architectures, and human experts, followed by validation on a prospective dataset of 188 lesions.ResultsOn the internal test set, the hybrid CNN-Transformer achieved an accuracy with 0.960, a sensitivity with 0.967, and an Area Under the Curve (AUC) with 0.9788. On the external dataset, it demonstrated robust generalization with 0.940 accuracy and 0.9730 AUC. Prospective validation confirmed these findings with an accuracy of 0.952 and an AUC of 0.9801. The proposed model outperforms the single-image baseline with 0.881 internal accuracy and 0.849 accuracy of a senior ultrasound physician.ConclusionThe system combines CNNs with Transformers to analyze multi-view ultrasound images, achieving high diagnostic accuracy and robustness. By fusing multi-view information, this model aligns with clinical logic and outperforms both single-image methods and human experts.