Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external...

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Infectious Disease Diagnostics

Background:

• Artificial intelligence (AI) tools are increasingly used for medical image analysis.
• Ali-M3 is an AI program designed to assess the likelihood of coronavirus disease (COVID-19) from chest computed tomography (CT) scans.
• External validation of AI tools is crucial for assessing their real-world clinical utility.

Purpose of the Study:

• To externally validate the accuracy of the Ali-M3 AI program for detecting COVID-19.
• To evaluate the clinical value of Ali-M3 in a Japanese patient cohort.

Main Methods:

• A retrospective cohort study was conducted using data from 617 symptomatic patients across 11 Japanese tertiary care facilities.
• Patients underwent reverse transcription-polymerase chain reaction (RT-PCR) testing and chest CT.
• Ali-M3 was used to determine COVID-19 infection probabilities, and its diagnostic performance was assessed using area under the curve (AUC), sensitivity, and specificity.

Main Results:

• The overall AUC for Ali-M3 in predicting COVID-19 was 0.797. At a probability cut-off of 0.5, sensitivity was 80.6% and specificity was 68.3%.
• A lower cut-off of 0.2 improved sensitivity to 89.2% but decreased specificity to 43.2%.
• For patients requiring oxygen, the AUC was 0.825, with high sensitivity (88.7% at 0.5 cut-off, 97.9% at 0.2 cut-off). Sensitivity improved after 5 days from symptom onset.

Conclusions:

• Ali-M3 demonstrated sufficient sensitivity for detecting COVID-19 in external validation, although specificity was lower.
• The AI tool may be valuable for excluding a diagnosis of COVID-19, particularly in conjunction with clinical assessment.
• Further evaluation may be warranted to optimize its use in clinical practice.