Doctor's Orders-Why Radiologists Should Consider Adjusting Commercial Machine Learning Applications in Chest...

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Radiology

Background:

• Deep learning (DL) software is increasingly being evaluated for medical image analysis.
• Chest radiographs are a common diagnostic tool requiring efficient and accurate interpretation.

Purpose of the Study:

• To evaluate the performance of a commercial deep learning software for chest radiograph interpretation.
• To explore the software's accuracy, sensitivity, and specificity across different diseases and patient groups.

Main Methods:

• Retrospective study of 477 patients' chest radiographs.
• Comparison of DL software readings against a gold standard established by two radiologists.
• Receiver Operating Characteristic (ROC) analysis to determine performance metrics.

Main Results:

• Overall AUC of 0.84, with 85% sensitivity and 75.4% specificity.
• Highest performance for pleural effusion (AUC 0.92, 86.4% sensitivity/specificity).
• Significant influence of sex, age, and comorbidity on DL performance; potential bias in elderly and female patients.

Conclusions:

• Commercial DL software demonstrates potential for autonomous chest radiograph reporting but currently requires human supervision.
• DL tools may be useful for ruling out specific conditions in screening scenarios, offering workload reduction.
• Radiologists should be aware of potential biases and adjust DL thresholds for optimal deployment.