Evaluating AI Models for Pneumothorax Detection on Chest Radiographs: Diagnostic Accuracy and Clinical Trade-Offs

Background Pneumothorax is a critical condition where timely recognition on chest radiographs is essential, particularly in emergency and resource-limited settings. Emerging artificial intelligence (AI) systems capable of native image interpretation offer potential to augment clinical workflows, yet their diagnostic reliability remains underexplored. Methods We evaluated two state-of-the-art AI models on 2,000 publicly available frontal chest radiographs, equally divided between pneumothorax-positive and pneumothorax-negative cases. Models were prompted with standardized diagnostic instructions emphasizing pleural line visualization, asymmetry in lung translucency, and the deep sulcus sign. Predictions were assessed against reference diagnoses using accuracy, precision, recall, and F1 score. Results One model achieved balanced diagnostic accuracy (64%) with a precision of 66% and a recall of 57%, while the other demonstrated higher sensitivity (88%) but lower precision (55%). These divergent profiles underscore trade-offs between minimizing false negatives and limiting false positives. Conclusions AI systems show promise for pneumothorax detection on chest radiographs but exhibit distinct diagnostic biases that must be carefully matched to the clinical context. Balanced performance models may be suitable for general screening, whereas high-sensitivity models may better support triage workflows. Rigorous validation, integration strategies, and human supervision remain essential before deployment in real-world clinical practice.