Biased Deep Learning Methods in Detection of COVID-19 Using CT Images: A Challenge Mounted by Subject-Wise-Split...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer-Aided Diagnosis

Background:

• Deep learning (DL) models are crucial for detecting respiratory system damage, including COVID-19, using CT images.
• Existing studies often report unreliable accuracy and lack repeatability due to slice-wise data splitting, creating dependencies between training and test sets.
• This issue compromises the real-world performance validation of DL models in medical diagnostics.

Purpose of the Study:

• To introduce a novel CT image dataset (ISFCT Dataset) with subject-wise splitting for unbiased DL model training and testing.
• To evaluate the actual performance of existing DL models using a subject-wise split, addressing the limitations of slice-wise validation.
• To explore the impact of data splitting strategies on model performance and repeatability in respiratory disease detection.

Main Methods:

• Development of the ISFCT Dataset with subject-wise labels for CT images, enabling robust DL model evaluation.
• Implementation of subject-wise data splitting to train and test DL algorithms, ensuring data independence between sets.
• Validation of previously published DL models on the ISFCT Dataset using subject-wise splits and comparison with slice-wise results.
• Utilizing t-distribution stochastic neighbor embedding (t-SNE) to visualize and demonstrate distribution differences between data splits.

Main Results:

• Reported high accuracies of existing DL models on slice-wise splits are not repeatable when using subject-wise splits.
• Significant distribution differences between slice-wise and subject-wise data splits were observed, impacting model generalization.
• Less complex DL models achieved competitive and repeatable results compared to complex models when trained and tested on subject-wise splits.
• The study demonstrates that complex DL models do not inherently guarantee accurate and repeatable performance.

Conclusions:

• Subject-wise data splitting is essential for unbiased evaluation and ensuring the repeatability of DL models in medical imaging, particularly for respiratory conditions.
• The ISFCT Dataset provides a valuable resource for developing and validating robust DL models for CT-based respiratory disease detection.
• Simpler DL models can be effective and reliable, challenging the notion that model complexity directly correlates with diagnostic accuracy and repeatability.