A ResNet-ViT classification model for thyroid nodules using ultrasound images and clinical information

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Thyroid nodules are common, and distinguishing benign from malignant types via ultrasound carries misdiagnosis risks.
• Computer-Aided Diagnosis (CAD) is essential for enhancing diagnostic accuracy and patient outcomes.

Purpose of the Study:

• To develop and evaluate a novel deep learning model, ResNet-ViT, for improved prediction of thyroid nodule malignancy.
• To assess the efficacy of a multimodal approach combining image and clinical data.

Main Methods:

• A hybrid ResNet-ViT model was designed, integrating ResNet18 for local features and Vision Transformer (ViT)-l-16 for global features.
• The ResNet-ViT model was further enhanced by fusing its image features with clinical data using the MCB algorithm, creating a multimodal model.
• Performance was evaluated on the DDTI and TN5000 datasets using 5-fold cross-validation and compared against eight other established models.

Main Results:

• The ResNet-ViT model demonstrated superior predictive performance compared to eight other methods.
• The ResNet-ViT multimodal model achieved significantly improved diagnostic accuracy, outperforming all other models.
• Incorporating surrounding tissue information in datasets enhanced the model's ability to differentiate between benign and malignant nodules.

Conclusions:

• The ResNet-ViT multimodal model effectively integrates both imaging and clinical features, offering a stable and accurate method for thyroid nodule classification.
• This advanced CAD approach holds significant potential for improving the diagnosis of thyroid nodules, reducing misdiagnosis rates, and benefiting patient care.