AI as teacher: effectiveness of an AI-based training module to improve trainee pediatric fracture detection

Prior work has demonstrated that AI access can help residents more accurately detect pediatric fractures. We wished to evaluate the effectiveness of an unsupervised AI-based training module as a pediatric fracture detection educational tool.

Two hundred forty radiographic examinations from throughout the pediatric upper extremity were split into two groups of 120 examinations. A previously developed open-source deep learning fracture detection algorithm ( www.childfx.com ) was used to annotate radiographs. Four medical students and four PGY-2 radiology residents first evaluated 120 examinations for fracture without AI assistance and subsequently reviewed AI annotations on these cases via a training module. They then interpreted 120 different examinations without AI assistance. Pre- and post-intervention fracture detection accuracy was evaluated using a chi-squared test.

Overall resident fracture detection accuracy significantly improved from 71.3% pre-intervention to 77.5% post-intervention (p = 0.032). Medical student fracture detection accuracy was not significantly changed from 56.3% pre-intervention to 57.3% post-intervention (p = 0.794). Eighty-eight percent of responding participants (7/8) would recommend this model of learning.

We found that a tailored AI-based training module increased resident accuracy for detecting pediatric fractures by 6.2%. Medical student accuracy was not improved, likely due to their limited background familiarity with the task. AI offers a scalable method for automatically generating annotated teaching cases covering varied pathology, allowing residents to efficiently learn from simulated experience.