Are Artificial Intelligence Models Reliable for Clinical Application in Pediatric Fracture Detection on Radiographs?...

Artificial intelligence (AI) applications for pediatric fracture diagnosis using radiographs have demonstrated growing potential in clinical settings. Despite this growing potential, existing studies are limited by small sample sizes, variability in their diagnostic metrics, and inconsistent use of external validation, which reduces confidence in their findings. These limitations hinder the assessment of real-world performance. A meta-analysis would help address these gaps by pooling data to generate more robust, generalizable estimates for clinical application and future guidance.

(1) What is the pooled diagnostic performance of AI models, including sensitivity, specificity, and area under the curve (AUC), for detecting pediatric fractures on radiographs? (2) What is the clinical applicability of AI models, as determined by whether their diagnostic performance is sustained in studies that employed external validation? (3) How does anatomic coverage influence the diagnostic performance of AI models?

This meta-analysis adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines and was registered in PROSPERO (CRD42024628342). A systematic search of PubMed/MEDLINE, Embase, and the Cochrane Library was conducted from database inception through December 9, 2024. A total of 497 records were identified. Eligible studies included pediatric patients with suspected fractures evaluated by AI models on radiographs. Studies were excluded if they lacked sufficient data to calculate sensitivity, specificity, or AUC; if they combined adult and pediatric populations; or if they focused on rib fractures. Sixteen diagnostic accuracy studies were included, involving 10,203 pediatric patients with a mean age of 8.85 years, 54% of whom were male, and 21,789 radiographs, of which 5882 confirmed fractures. Data extraction followed the Population, Index test, Target condition (PIT) framework and was performed independently by two reviewers. The risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool, which evaluates four domains (patient selection, index test, reference standard, and flow/timing) for low, high, or unclear risk. Most studies exhibited low to moderate risk of bias. Certainty of evidence was evaluated using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach, which classifies evidence as high, moderate, low, or very low, and in this study demonstrated high certainty of evidence. Heterogeneity in the pooled estimates was moderate for sensitivity (I 2 = 61%) and high for specificity (I 2 = 90%). No evidence of publication bias was detected based on Egger test (p = 0.54) and funnel plot symmetry. Meta-analyses used logit transformation and bivariate modeling to estimate pooled sensitivity, specificity, and AUC.

The pooled analysis demonstrated a sensitivity of 93% (95% confidence interval [CI] 92% to 94%), a specificity of 91% (95% CI 88% to 93%), and an AUC of 0.96 (95% CI 0.92 to 0.97). The AUC reflects the overall ability of a model to distinguish between patients with and without fractures, with values closer to 1.0 indicating better diagnostic performance. When evaluated on external data sets, AI models maintained high diagnostic accuracy, with a sensitivity of 93% (95% CI 90% to 95%), specificity of 88% (95% CI 84% to 91%), and an AUC of 0.95 (95% CI 0.89 to 0.97), supporting their potential for clinical applicability. Anatomic coverage by specific region made a meaningful contribution to explaining the observed heterogeneity. Models evaluating multiple regions showed slightly higher sensitivity, while those focused on single regions demonstrated better specificity, suggesting that a broader anatomic scope may improve fracture detection but slightly reduce accuracy in ruling out false positives.

This meta-analysis demonstrates that AI models can accurately detect pediatric fractures on radiographs, a finding that withstood scrutiny in studies that included external validation. These findings suggest that orthopaedic surgeons and emergency physicians can consider incorporating validated convolutional neural network algorithms into workflows to enhance diagnostic accuracy, especially in acute care settings where rapid and accurate decision-making is critical. Nevertheless, future research is needed to investigate performance across specific subgroups, including sex and anatomic regions. Paired-design diagnostic accuracy studies with external geographic validation remain the most appropriate method to assess their real-world value. Such validation should be prioritized as a prerequisite for clinical generalization and democratization of AI models, even before randomized trials or prospective implementation studies.

Level III, diagnostic study.