Deep Learning Systems for Pneumothorax Detection on Chest Radiographs: A Multicenter External Validation Study.

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology and Diagnostic Imaging
• Machine Learning for Healthcare

Background:

• Pneumothorax detection from chest radiographs is critical for patient care.
• Deep learning models offer potential for automated detection but require validation across diverse datasets.
• Assessing generalizability is crucial for clinical implementation of AI tools.

Purpose of the Study:

• To evaluate the performance of a deep learning model for pneumothorax detection on external, multi-institutional datasets.
• To identify patient and acquisition factors influencing the model's detection accuracy.
• To determine the generalizability of AI-assisted pneumothorax diagnosis.

Main Methods:

• A deep learning model was trained on combined ChestX-ray14 and CheXpert datasets.
• The model was retrospectively tested on six independent external datasets (A-F).
• Performance was assessed using AUC, sensitivity, specificity, and predictive values, with radiologist consensus as the reference standard.

Main Results:

• The model achieved high AUCs across external datasets, ranging from 0.91 to 0.98, comparable to the internal AUC of 0.93.
• Performance was significantly better for large pneumothoraces (AUC 0.96) compared to small ones (AUC 0.88, P = .005).
• Model performance was not affected by the presence or absence of a chest tube (P > .99).

Conclusions:

• The deep learning model generalized well to diverse external datasets, indicating robustness.
• The model's performance is largely independent of patient demographics and technical acquisition parameters.
• Further refinement may be needed to optimize detection of small pneumothoraces.