Multitask Swin Transformer for classification and characterization of pulmonary nodules in CT images

Early diagnosis of pulmonary nodules is essential for effective prevention and treatment of pulmonary cancer. However, the heterogeneous and complex characteristics of pulmonary nodules, such as shape, size, speculation, and texture, present significant challenges in clinical diagnosis, which computer-aided diagnosis (CAD) can help address. Moreover, the varied performance of deep learning methods in CAD and limited model interpretability often hinder clinicians' understanding of CAD results. In this study, we propose a multitask Swin Transformer (MTST) for classifying benign and malignant pulmonary nodules, which outputs nodule features as classification criteria.

We introduce a MTST model for feature extraction, designed with a multitask layer that simultaneously outputs benign and malignant binary classification, multilevel classification, and a detailed analysis of pulmonary nodule features. In addition, we incorporate image augmentation using a U-Net generative adversarial network (GAN) model to enhance the training process.

Experimental findings on the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset demonstrate that the proposed MTST outperforms conventional convolutional neural networks (CNN)-based networks across multiple tasks. Specifically, MTST achieved an accuracy of 93.24% in binary classification of benign and malignant nodules and demonstrated superior performance in nodule feature evaluation. For multilevel classification of pulmonary nodules, the Swin Transformer achieved an accuracy of 95.73%. On the training, validation, and test sets (9,600/2,400/1,600 nodules), the MTST model achieved an accuracy of 93.74%, sensitivity of 91.55%, and specificity of 96.09%. The results indicate that the MTST model aligns well with clinical diagnostic practices, offering improved performance and reliability.

The MTST model's efficacy in binary classification, multiclass classification, and feature evaluation confirms its potential as a valuable tool for CAD systems in clinical settings.