Comparison of CNNs and Transformer Models in Diagnosing Bone Metastases in Bone Scans Using Grad-CAM

Area of Science:

• Medical Imaging
• Artificial Intelligence in Oncology
• Deep Learning for Bone Metastasis Detection

Background:

• Convolutional neural networks (CNNs) are used for detecting bone metastases on bone scans.
• The performance of newer models like ConvNeXt and transformer architectures in this application is not well-established.

Purpose of the Study:

• To evaluate the diagnostic performance of various deep learning models, including ConvNeXt and transformer models, for detecting bone metastases on bone scans.
• To compare the efficacy of these models against established CNNs.

Main Methods:

• Retrospective analysis of 2 institutional bone scan datasets (n=4626 training/validation, n=1428 test).
• Evaluation of ResNet18, Data-Efficient Image Transformer (DeiT), Vision Transformer (ViT Large 16), Swin Transformer (Swin Base), and ConvNeXt Large models.
• Gradient-weighted class activation mapping (Grad-CAM) for model visualization.

Main Results:

• ConvNeXt Large achieved the highest performance (validation: 0.969, test: 0.885), outperforming ResNet (validation: 0.892, test: 0.725).
• Swin Base also showed strong performance (validation: 0.965, test: 0.840), significantly better than ResNet.
• All models performed better in detecting polymetastasis than oligometastasis. ConvNeXt focused on local lesions, Swin Base on global areas.

Conclusions:

• ConvNeXt demonstrates superior diagnostic performance for bone metastasis detection on bone scans compared to traditional CNNs and transformer models.
• The ConvNeXt model shows significant potential for enhancing medical image analysis in cancer diagnostics, particularly for poly-metastatic cases.