A cognitive IoT-based framework for effective diagnosis of COVID-19 using multimodal data

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Deep Learning

Background:

• COVID-19 pandemic necessitates accurate diagnostic tools.
• Single-modality prediction methods for COVID-19 have limitations in accuracy.
• Multi-modal data integration can improve diagnostic precision and reduce misdiagnosis.

Purpose of the Study:

• To develop a robust IoT and deep learning-based multi-modal classification method for accurate COVID-19 prediction.
• To introduce lightweight deep learning models for unimodal COVID-19 classification.
• To create a dynamic multi-modal fusion classifier for enhanced diagnostic performance.

Main Methods:

• Development of CovParaNet (audio classification) and CovTinyNet (image classification) lightweight deep learning models.
• Comparative analysis of unimodal models against benchmarks.
• Integration of five independently trained unimodal models using a dynamic multi-modal Random Forest classifier.
• Implementation of an online retraining mechanism for adaptability.

Main Results:

• CovParaNet achieved 97.45% accuracy for audio classification.
• CovTinyNet achieved 99.19% accuracy for image classification.
• The proposed dynamic multi-modal fusion model demonstrated 100% accuracy, precision, and recall.
• The system maintained high reliability and effectiveness even with missing input modalities.

Conclusions:

• The proposed multi-modal deep learning approach significantly enhances COVID-19 prediction accuracy.
• Lightweight models and dynamic fusion offer efficient and reliable diagnostic solutions.
• The system's robustness and adaptability support its use in diverse and challenging environments.