Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Ultrasound Technology

Background:

• Deep learning (DL) models for medical image classification often fail to generalize to external datasets.
• Limited clinical data availability hinders comprehensive assessment of model performance in subgroups.
• A single-center DL model was developed to detect lung sliding artifact on lung ultrasound (LUS).

Purpose of the Study:

• To validate a lung sliding artifact detection model using external LUS data from multiple institutions.
• To optimize the use of scarce external data for improving DL model generalizability.
• To identify the minimum data required for achieving predefined performance goals using a novel fine-tuning technique.

Main Methods:

• External LUS data from three tertiary care centers (641 clips, 238 patients) were used for validation.
• The novel Threshold-Aware Accumulative Fine-Tuning (TAAFT) method was employed to fine-tune the baseline model.
• Subgroup analysis and Grad-CAM++ explanations were utilized to assess model performance and interpretability.

Main Results:

• The fine-tuned model achieved 0.917 sensitivity, 0.817 specificity, and 0.920 AUC on the external validation dataset.
• Performance goals were exceeded, demonstrating improved generalizability.
• Subgroup analyses revealed specific LUS characteristics challenging the model, and Grad-CAM++ highlighted relevant image regions.

Conclusions:

• A multicenter study successfully improved a lung sliding DL model's generalizability and performance using limited external data.
• The TAAFT method offers an efficient approach for DL researchers working with smaller external validation datasets.
• Identifying poorly performing subgroups informs future iterative model improvements.