Related Experiment Video
Updated: Jun 27, 2025
DNA Virus Detection System Based on RPA-CRISPR/Cas12a-SPM and Deep Learning
Published on: May 10, 2024
COVID-19 Hierarchical Classification Using a Deep Learning Multi-Modal
Albatoul S Althenayan1,2, Shada A AlSalamah1,3,4, Sherin Aly5
1Information Systems Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia.
Insights
This study developed an accurate deep learning model using chest X-ray images and medical data to diagnose COVID-19 pneumonia, distinguishing it from other lung conditions. The multi-modal approach enhances radiological diagnostics for faster patient isolation and treatment.
Area of Science:
- Radiology and Medical Imaging
- Artificial Intelligence in Healthcare
- Deep Learning for Disease Diagnosis
Background:
- Coronavirus disease 2019 (COVID-19) presents diagnostic challenges due to time constraints, limited radiologist availability, and RT-PCR limitations.
- Existing deep learning models for COVID-19 classification often use binary methods, single modalities, small datasets, and neglect the hierarchical nature of pneumonia classification.
- Accurate and timely diagnosis of COVID-19 is crucial for patient isolation and effective public health management.
Purpose of the Study:
- To develop a multi-modal deep learning approach for accurate COVID-19 pneumonia identification and differentiation from other pneumonia types and healthy lungs.
- To demonstrate the value of integrating chest X-ray (CXR) images with tabular medical data for improved diagnostic accuracy.
- To address limitations of previous studies by utilizing a hierarchical classification structure and generating synthetic data for imbalanced datasets.
Main Methods:
- Employed Resnet-based and VGG-based pre-trained convolutional neural network (CNN) models for feature extraction from CXR images.
- Utilized early fusion to combine features from CNN models and tabular medical data for classifying eight distinct classes.
- Leveraged a hierarchical classification structure and generative adversarial networks (GANs) to handle imbalanced datasets and improve classification outcomes.
Main Results:
- The proposed multi-modal deep learning approach achieved a macro-average F1-score of 95.9% on private datasets.
- COVID-19 identification specifically reached an F1-score of 87.5% using a Resnet-based structure.
- The integration of tabular medical data significantly contributed to the enhanced diagnostic accuracy of the model.
Conclusions:
- A highly accurate deep learning multi-modal system was created for diagnosing COVID-19 and differentiating it from other pneumonia types and normal lung conditions.
- The developed model effectively enhances the radiological diagnostic process, enabling more timely and accurate identification of COVID-19.
- This research highlights the potential of combining imaging and clinical data with advanced AI techniques for robust disease diagnosis.
Abstract:
Coronavirus disease 2019 (COVID-19), originating in China, has rapidly spread worldwide. Physicians must examine infected patients and make timely decisions to isolate them. However, completing these processes is difficult due to limited time and availability of expert radiologists, as well as limitations of the reverse-transcription polymerase chain reaction (RT-PCR) method. Deep learning, a sophisticated machine learning technique, leverages radiological imaging modalities for disease diagnosis and image classification tasks. Previous research on COVID-19 classification has encountered several limitations, including binary classification methods, single-feature modalities, small public datasets, and reliance on CT diagnostic processes. Additionally, studies have often utilized a flat structure, disregarding the hierarchical structure of pneumonia classification. This study aims to overcome these limitations by identifying pneumonia caused by COVID-19, distinguishing it from other types of pneumonia and healthy lungs using chest X-ray (CXR) images and related tabular medical data, and demonstrate the value of incorporating tabular medical data in achieving more accurate diagnoses. Resnet-based and VGG-based pre-trained convolutional neural network (CNN) models were employed to extract features, which were then combined using early fusion for the classification of eight distinct classes. We leveraged the hierarchal structure of pneumonia classification within our approach to achieve improved classification outcomes. Since an imbalanced dataset is common in this field, a variety of versions of generative adversarial networks (GANs) were used to generate synthetic data. The proposed approach tested in our private datasets of 4523 patients achieved a macro-avg F1-score of 95.9% and an F1-score of 87.5% for COVID-19 identification using a Resnet-based structure. In conclusion, in this study, we were able to create an accurate deep learning multi-modal to diagnose COVID-19 and differentiate it from other kinds of pneumonia and normal lungs, which will enhance the radiological diagnostic process.
More Related Videos
Related Concept Videos
01:17Classification of Illness
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
01:26Classification of Systems-I
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
01:31Classification of Systems-II
01:29Aggregates Classification
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
01:30Classification of Leukocytes
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
01:30Classification of Signals
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...