Multimodal Deep Learning Network for Differentiating Between Benign and Malignant Pulmonary Ground Glass Nodules

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Pulmonary ground glass nodules (GGNs) require accurate differentiation between benign and malignant types for effective patient management.
• Current diagnostic methods may have limitations in achieving high accuracy for GGN classification.

Purpose of the Study:

• To develop and evaluate a multimodal deep learning network model for enhanced diagnosis of benign versus malignant pulmonary GGNs.

Main Methods:

• A retrospective dataset of pulmonary GGNs from multiple Chinese centers was utilized, split into training, validation, and test sets.
• A multimodal classification model was constructed using Residual Network (ResNet) for imaging data, Word2Vec for semantic information, and Self Attention for feature integration.
• The model's diagnostic performance was compared against VGG and ResNet models and human radiologists.

Main Results:

• The multimodal model achieved 90.2% accuracy in the validation set, outperforming VGG and ResNet models.
• In the test set, the model showed 91.18% accuracy for malignant GGNs and 80.70% for benign GGNs, exceeding radiologists' performance in malignant GGN diagnosis.
• The model demonstrated strong agreement with postoperative pathology (Kappa=0.720), indicating superior alignment with gold standard findings compared to radiologists.

Conclusions:

• The developed multimodal deep learning model shows significant potential in accurately diagnosing benign and malignant GGNs.
• This model can serve as a valuable reference tool for radiologists, potentially improving diagnostic accuracy and work efficiency in clinical practice.