Real-World Integration of an Automated Tool for Intracranial Hemorrhage Detection in an Unselected Cohort of...

Background/Objectives: Intracranial hemorrhage (ICH) is a life-threatening condition that can be rapidly detected by non-contrast head computed tomography (NCCT). RAPID ICH is a deep learning (DL) tool for automatic ICH identification using NCCT. Our aim was to assess the real-world performance of RAPID ICH compared to that of a first-year radiology resident on consecutively acquired NCCTs from patients referred from the Emergency Department. Methods: This single-center retrospective cohort study included NCCTs acquired on the same CT scanner over three months. Exclusion criteria were motion or metallic artifacts that substantially degraded the NCCT quality and incomplete NCCTs. Two senior neuroradiologists conducted ground-truth labeling of the NCCTs regarding ICH presence in a binary manner. The first-year radiology resident assessed NCCTs for ICH presence and was blinded to the ground-truth labeling. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were computed for the RAPID ICH identifications and for the first-year radiology resident's ICH identifications. Results: After applying exclusion criteria, 844 NCCTs remained. Ground-truth labeling found ICH in 63 NCCTs. RAPID ICH showed 87.3% sensitivity, 74% specificity, 21.3% PPV, and 98.6% NPV, while the first-year radiology resident achieved 95.2% sensitivity, 90.8% specificity, 45.5% PPV, and 99.6% NPV. There were 8 false-negative and 203 false-positive RAPID ICH identifications. Conclusions: RAPID ICH's sensitivity and specificity were lower than in prior studies performed using RAPID ICH, and there was a high number of false-positive RAPID ICH identifications, limiting the generalizability of the assessed version of this DL tool. Testing DL tools by comparing them with radiologists of varying experience can provide valuable insights into their performance.