Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Convolutional Neural Networks (CNNs) excel at localized feature extraction in breast ultrasound (BUS) but struggle with global context.
• Vision Transformers capture global information but may alter local image details through tokenization.
• Accurate BUS tumor classification is vital for diagnosis and treatment planning.

Purpose of the Study:

• To develop and evaluate a hybrid deep neural network, Hybrid-MT-ESTAN, for multitask BUS tumor classification and segmentation.
• To address the limitations of pure CNNs and Vision Transformers in capturing both local and global image features.
• To enhance the diagnostic performance in BUS image analysis.

Main Methods:

• Proposed a hybrid multitask deep neural network (Hybrid-MT-ESTAN) integrating CNNs and Swin Transformer components.
• Employed a hybrid architecture to leverage the strengths of both CNNs for local patterns and Swin Transformers for global context.
• Evaluated the model on a dataset of 3,320 BUS images using seven quantitative metrics.

Main Results:

• Hybrid-MT-ESTAN achieved the highest accuracy (82.7%), sensitivity (86.4%), and F1 score (86.0%) compared to nine other BUS classification methods.
• Demonstrated superior performance in BUS tumor classification, outperforming existing approaches.
• The hybrid architecture effectively balanced the modeling of local and global contextual information.

Conclusions:

• The proposed Hybrid-MT-ESTAN model offers a significant advancement in breast ultrasound image analysis.
• Hybrid architectures combining CNNs and Swin Transformers are effective for BUS tumor classification and segmentation.
• This approach holds promise for improving diagnostic accuracy in breast cancer detection.