External validation of a commercially available deep learning algorithm for fracture detection in children

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Pediatric Radiology

Background:

• Deep learning algorithms are increasingly used for medical image analysis.
• Accurate fracture detection in children is crucial for timely treatment and management.
• External validation of AI tools in real-world clinical settings is essential.

Purpose of the Study:

• To externally validate the Rayvolve® deep learning algorithm for pediatric fracture detection.
• To assess the algorithm's performance on a real-life cohort of children presenting to the emergency room.
• To evaluate diagnostic performance across detection, enumeration, and localization approaches.

Main Methods:

• Retrospective analysis of 2634 radiography sets (5865 images) from 2549 children.
• Comparison of Rayvolve® algorithm's fracture assessment against senior radiologist diagnoses.
• Performance evaluation using detection, enumeration, and localization approaches.
• Subgroup analyses based on cast presence, age category (0-4 vs. 5-18 years), and anatomical region.

Main Results:

• The algorithm achieved 95.7% sensitivity, 91.2% specificity, and 92.6% accuracy in the detection approach.
• For enumeration and localization, sensitivity was 94.1%, specificity 88.8%, and accuracy 90.4%.
• Performance was significantly better in children aged 5-18 years compared to 0-4 years.
• High negative predictive value was consistent across subgroups, except in patients with casts.

Conclusions:

• The Rayvolve® deep learning algorithm is a reliable tool for pediatric fracture detection.
• The algorithm shows particular efficacy in older children (over 4 years) and those without casts.
• Further validation may be needed for pediatric patients with casts.