Real-life performance of AI-aided radiologists, emergency physicians and two AI solutions for diagnosing bone...

To compare the performance of artificial intelligence (AI)-aided radiologists, emergency physicians and two AI solutions for diagnosing bone fractures.

Consecutive patients treated at two centres for appendicular skeletal traumatic injury between January and April 2021 who underwent X-ray imaging and whose initial conclusions were available and prospectively encoded by emergency physicians, were also prospectively analysed via two AI solutions (BoneView and SmartUrgence) available for the real-life interpretation of AI-aided radiologists. The ground truth was retrospectively assessed by 5 senior musculoskeletal radiologists who were aware of all the clinical, radiological and AI data. Numbers of suspected fractures, true positives and false positives per AI were compared. Diagnostic performance metrics (sensitivity, specificity, positive and negative predictive values and accuracy with 95% confidence intervals) for detecting fractures were estimated for each interpretation (emergency physician, BoneView, SmartUrgence, AI-aided radiologist).

969 patients with 1049 radiography sets were included, 287 of whom had fractures (27.4 %). The average number of any fracture and true positive fractures were greater with BoneView than with SmartUrgence (P = 0.0469 and P = 0.0022, respectively). The real-life sensitivity, specificity and accuracy for detecting fracture in the entire cohort were 93 %, 99 % and 97.6 % for AI-aided radiologists; 80.8 %, 97.6 % and 93 % for emergency physicians; 89.5 %, 93.8 % and 92.7 % for BoneView; and 85.7 %, 94.6 % and 92.2 % for SmartUrgence.

In a real-life emergency setting, the performance of AI-aided radiologists in diagnosing bone fractures was excellent, and these radiologists outperformed AI solutions alone regardless of age and location.