External Validation of a Winning Artificial Intelligence Algorithm from the RSNA 2022 Cervical Spine Fracture...

Area of Science:

• Radiology
• Medical Imaging
• Artificial Intelligence in Medicine

Background:

• The Radiological Society of North America (RSNA) has supported AI challenges since 2017.
• Recent RSNA 2022 Cervical Spine Fracture Detection Challenge algorithms showed high performance on the competition dataset.
• Real-world clinical performance of these algorithms remains unassessed.

Purpose of the Study:

• To conduct a generalizability test of a leading AI algorithm from the RSNA 2022 challenge.
• To assess the feasibility of using these AI models in clinical practice.

Main Methods:

• A deep learning algorithm was selected for its performance, portability, and ease of use.
• The algorithm was locally installed and tested on 100 cervical spine CT scans (50 fractures, 50 negative) from a Level 1 trauma center.
• Ground truth was established via radiology reports; sensitivity, specificity, F1 score, and AUC were calculated.

Main Results:

• The external validation group was older than the competition group (53.5 ± 21.8 vs 58 ± 22.0 years; p < .05).
• Sensitivity was 86% and specificity was 70% in the external group, compared to 85% and 94% in the competition group.
• Misclassified fractures often involved advanced degenerative disease, subtle nondisplaced fractures, or malalignment.

Conclusions:

• The AI model demonstrated similar sensitivity on the test and external datasets, suggesting potential as an emergency triage tool.
• Factors like age-associated comorbidities may impact AI model accuracy and specificity in certain populations.
• Further research is needed to understand the contributions and limitations of AI algorithms in clinical settings.