COVID-19 Hierarchical Classification Using a Deep Learning Multi-Modal

Area of Science:

• Radiology and Medical Imaging
• Artificial Intelligence in Healthcare
• Deep Learning for Disease Diagnosis

Background:

• Coronavirus disease 2019 (COVID-19) presents diagnostic challenges due to time constraints, limited radiologist availability, and RT-PCR limitations.
• Existing deep learning models for COVID-19 classification often use binary methods, single modalities, small datasets, and neglect the hierarchical nature of pneumonia classification.
• Accurate and timely diagnosis of COVID-19 is crucial for patient isolation and effective public health management.

Purpose of the Study:

• To develop a multi-modal deep learning approach for accurate COVID-19 pneumonia identification and differentiation from other pneumonia types and healthy lungs.
• To demonstrate the value of integrating chest X-ray (CXR) images with tabular medical data for improved diagnostic accuracy.
• To address limitations of previous studies by utilizing a hierarchical classification structure and generating synthetic data for imbalanced datasets.

Main Methods:

• Employed Resnet-based and VGG-based pre-trained convolutional neural network (CNN) models for feature extraction from CXR images.
• Utilized early fusion to combine features from CNN models and tabular medical data for classifying eight distinct classes.
• Leveraged a hierarchical classification structure and generative adversarial networks (GANs) to handle imbalanced datasets and improve classification outcomes.

Main Results:

• The proposed multi-modal deep learning approach achieved a macro-average F1-score of 95.9% on private datasets.
• COVID-19 identification specifically reached an F1-score of 87.5% using a Resnet-based structure.
• The integration of tabular medical data significantly contributed to the enhanced diagnostic accuracy of the model.

Conclusions:

• A highly accurate deep learning multi-modal system was created for diagnosing COVID-19 and differentiating it from other pneumonia types and normal lung conditions.
• The developed model effectively enhances the radiological diagnostic process, enabling more timely and accurate identification of COVID-19.
• This research highlights the potential of combining imaging and clinical data with advanced AI techniques for robust disease diagnosis.