Challenges of deep learning methods for COVID-19 detection using public datasets.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer-Aided Diagnosis

Background:

• Deep Learning (DL) models show promise for automated COVID-19 detection from medical images.
• High accuracies reported in prior studies often lack validation on independent test sets, raising concerns about overfitting and dataset-specific artifacts.

Purpose of the Study:

• To investigate the challenges and limitations of current DL models and public datasets for COVID-19 detection.
• To assess the impact of data source diversity and independence on DL model performance.
• To propose requirements for improved datasets for clinical application.

Main Methods:

• A Convolutional Neural Network (CNN) based model, CVR-Net (COVID-19 Recognition Network), was developed as a multi-scale-multi-encoder ensemble.
• Experiments were conducted using single, multiple, and independent data sources for 2-, 3-, and 4-class classification tasks.
• Model performance was evaluated based on classification accuracy across different data partitioning strategies.

Main Results:

• Binary classification accuracy reached 99.8% with a single train-test data source.
• Accuracy dropped significantly to 98.4% with multiple sources and 88.7% with independent sources.
• Similar performance degradation was observed in multi-class classification tasks, confirming the impact of data independence.

Conclusions:

• Existing public datasets and DL models for COVID-19 detection may not generalize well to independent data due to overfitting.
• The development of reliable DL tools for clinical settings necessitates datasets with independent test sets and balanced representation across diverse sources.
• Future datasets should include varied hospital and demographic data to ensure broader applicability and robustness.