Deep Learning-Based COVID-19 Pneumonia Classification Using Chest CT Images: Model Generalizability.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Radiology

Background:

• The COVID-19 pandemic spurred AI research for disease identification using medical data.
• Concerns exist regarding the generalizability of AI models due to heterogeneous training data.

Purpose of the Study:

• To evaluate the generalizability of deep learning (DL) classification models for COVID-19 detection.
• To assess model performance on 3D computed tomography (CT) datasets from diverse international sources.

Main Methods:

• Trained nine DL models on combined and single-source CT datasets (UTSW, CC-CCII, COVID-CTset, MosMedData).
• Evaluated models on internal and external test sets using accuracy and Area Under the Receiver Operating Characteristic Curve (AUC).

Main Results:

• Models trained on single datasets performed well internally (AUC up to 0.988).
• Performance drastically dropped to near random chance (AUC ~0.5) when models were tested on external, unseen datasets.
• Inclusion of a positive-only dataset (MosMedData) did not consistently improve performance.

Conclusions:

• Current DL models for COVID-19 detection on CT scans lack generalizability across different geographical cohorts.
• Data shift, influenced by patient demographics and imaging variations, significantly impacts model performance.
• Future research must prioritize diverse, multi-center datasets for robust AI model development in medical imaging.