Validation of a Deep Learning Tool for Detection of Incidental Vertebral Compression Fractures
Michelle Dai1,2, Bryan-Clement Tiu1, Jacob Schlossman1
1Irvine School of Medicine, University of California, Irvine, CA.
Objective:
This study evaluated the performance of a deep learning-based vertebral compression fracture (VCF) detection tool in patients with incidental VCF. The purpose of this study was to validate this tool across multiple sites and multiple vendors.
Methods:
This was a retrospective, multicenter, multinational blinded study using anonymized chest and abdominal CT scans performed for indications other than VCF in patients ≥50 years old. Images were obtained from 2 teleradiology companies in France and United States and were processed by CINA-VCF v1.0, a deep learning algorithm designed for VCF detection. Ground truth was established by majority consensus across 3 board-certified radiologists. Overall performance of CINA-VCF was evaluated, as well as subset analyses based on imaging acquisition parameters, baseline patient characteristics, and VCF severity. A subgroup was also analyzed and compared with available clinical radiology reports.
Results:
Four hundred seventy-four CT scans were included in this study, comprising 166 (35.0%) positive and 308 (65.0%) negative VCF cases. CINA-VCF demonstrated an area under the curve (AUC) of 0.97 (95% CI: 0.96-0.99), accuracy of 93.7% (95% CI: 91.1%-95.7%), sensitivity of 95.2% (95% CI: 90.7%-97.9%), and specificity of 92.9% (95% CI: 89.4%-96.5%). Subset analysis based on VCF severity resulted in a specificity of 94.2% (95% CI: 90.9%-96.6%) for grade 0 negative cases and a specificity of 64.3% (95% CI: 35.1%-87.2%) for grade 1 negative cases. For grades 2 and 3 positive cases, sensitivity was 89.7% (95% CI: 79.9%-95.8%) and 99.0% (95% CI: 94.4%-100.0%), respectively.
Conclusions:
CINA-VCF successfully detected incidental VCF and even outperformed clinical reports. The performance was consistent among all subgroups analyzed. Limitations of the tool included various confounding pathologies such as Schmorl's nodes and borderline cases. Despite these limitations, this study validates the applicability and generalizability of the tool in the clinical setting.
