Real-world performance and misclassification patterns of deep learning for intracranial hemorrhage detection

To evaluate the real-world diagnostic accuracy and systematic misclassification patterns of a deep learning algorithm (Hevi AI) for detecting intracranial hemorrhage (ICH) on non-contrast head computed tomography (CT) within a single-center cohort.

This retrospective, single-center study included 1,617 emergency non-contrast head CT examinations. Two expert radiologists served as the reference standard. The algorithm's binary detection performance (sensitivity, specificity, negative predictive value [NPV]) and subtyping accuracy were analyzed. Statistical analyses evaluated age and sex differences between false-positive (FP) and true-negative (TN) cases.

The reference standard ICH prevalence was 13.4% (n = 218). The algorithm demonstrated 98.17% sensitivity, 94.71% specificity and 99.70% NPV for hemorrhage detection. Specificity demonstrated an inverse relationship with patient age (96.95% in < 50 years, 93.04% in 50-70 years, and 88.54% in > 70 years). Among 74 FP cases, isolated subarachnoid hemorrhage (SAH) was the most frequent misclassification (62.16%). The mean age of FP cases (56.3 ± 22.3 years) was significantly higher than that of TN cases (40.5 ± 25.2 years) (p < 0.001). Subtyping errors (n = 96) predominantly showed an "over-labeling" pattern: 49 cases (51.0%) involved unnecessary epidural hemorrhage components added to subdural hemorrhages, and 31 cases (32.3%) involved unnecessary SAH additions.

While the Hevi AI algorithm demonstrates high sensitivity and an excellent NPV that support its utility in worklist prioritization, it exhibits specific neuroanatomical blind spots. Ischemic processes, vasogenic edema, and age-related structural changes are major sources of FP alerts. Furthermore, the algorithm systematically struggles to differentiate fine extra-axial boundaries, leading to over-labeling. Recognizing these specific failure patterns and geriatric variations is essential for safe clinical integration and accurate interpretation of AI-assisted neuroimaging.