Improved disease diagnosis system for COVID-19 with data refactoring and handling methods

Area of Science:

• Medical Informatics
• Computer Science
• Machine Learning

Background:

• The COVID-19 pandemic has highlighted the critical need for rapid and accurate diagnostic tools.
• Diagnostic inaccuracies can lead to significant psychological impacts and hinder effective disease control.
• Limited data availability poses a challenge for developing robust predictive models for infectious diseases.

Purpose of the Study:

• To develop an improved machine learning model for the diagnosis of COVID-19.
• To address the challenge of limited data for disease prediction models through advanced data handling techniques.
• To compare the performance of various machine learning algorithms for COVID-19 diagnosis.

Main Methods:

• Utilized the Hospital Israelita Albert Einstein dataset for COVID-19 diagnosis.
• Applied data preprocessing, feature ranking using distributed Decision Trees, and data augmentation techniques.
• Developed and evaluated several machine learning models including Random Forest, K-Nearest Neighbor, Logistic Regression, Support Vector Machine, and Deep Neural Network.
• Implemented all algorithms in a distributed environment using the Spark platform.

Main Results:

• The Deep Neural Network (DNN) model achieved the highest accuracy (96.99%), recall (96.98%), and precision (96.94%).
• The performance of the DNN model is comparable to or surpasses existing research in COVID-19 diagnosis.
• Feature ranking was performed using distributed Decision Trees to identify key diagnostic indicators.

Conclusions:

• Machine learning, particularly Deep Neural Networks, offers a promising approach for accurate and efficient COVID-19 diagnosis.
• Data augmentation and robust data handling techniques are crucial for improving model performance with limited datasets.
• The use of distributed computing platforms like Spark enables the efficient implementation of complex machine learning models for healthcare applications.