A SAM2 and VLM-Based Automatic Radiological Reporting System for Pneumothorax Detection

Background/Objectives: Pneumothorax, caused by air accumulation in the pleural space, can lead to lung collapse and, if diagnosis is delayed, respiratory or cardiovascular deterioration. Most deep-learning studies on pneumothorax address classification or segmentation alone. This study proposes a holistic automatic radiological reporting system combining U-Net-based segmentation, Segment Anything Model 2 (SAM2)-based boundary refinement, and a Vision-Language Model (VLM)-based reporting module for pneumothorax detection on computed tomography (CT) images. Methods: A MobileNetV2-encoder U-Net was trained on 224×224 CT patches from 171 patients (15,555 patches). A frozen, zero-shot SAM2 module, guided by a box prompt derived from the U-Net mask, was combined with U-Net via an Intersection over Union (IoU)-agreement rule. The refined mask and rule-based geometric features were passed to a frozen VLM (Qwen2-VL-2B-Instruct) to generate structured reports with a combined confidence coefficient. Performance was evaluated with Accuracy, Dice, IoU, and Loss, over all patches and over pneumothorax-positive patches only. Results: U-Net achieved 93.09/92.09/99.71% (Dice/IoU/Accuracy) on validation and 93.68/92.79/99.79% on the test set; these largely reflect correct empty-mask prediction on negative patches. On positive patches only, Dice was 0.6295 (validation) and 0.6215 (test), with patch-level sensitivity of 82.2% (validation) and 79.9% (test) and specificity above 97%. SAM2 refinement achieved 92.44% Dice and 91.29% IoU on a random test subsample (n ≈ 300), close to but not exceeding U-Net alone; an ablation analysis confirmed no consistent Dice/IoU or 95th-percentile Hausdorff distance (HD95) gain from SAM2. BLEU/ROUGE and Likert evaluation frameworks for the VLM reports were defined but not yet scored against reference reports. Conclusions: U-Net-based segmentation reliably identifies pneumothorax-containing patches with moderate boundary accuracy, while frozen SAM2 refinement adds limited value. The rule-based reporting layer is reproducible and shows triage-support potential, whereas the VLM free-text layer carries a documented hallucination risk and requires radiologist verification before clinical use.