Related Experiment Video
Updated: Sep 21, 2025
Automated Midline Shift and Intracranial Pressure Estimation based on Brain CT Images
Published on: April 13, 2013
Artificial Intelligence with Statistical Confidence Scores for Detection of Acute or Subacute Hemorrhage on
Eli Gibson1, Bogdan Georgescu1, Pascal Ceccaldi1
1Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
Purpose:
To present a method that automatically detects, subtypes, and locates acute or subacute intracranial hemorrhage (ICH) on noncontrast CT (NCCT) head scans; generates detection confidence scores to identify high-confidence data subsets with higher accuracy; and improves radiology worklist prioritization. Such scores may enable clinicians to better use artificial intelligence (AI) tools.
Materials And Methods:
This retrospective study included 46 057 studies from seven "internal" centers for development (training, architecture selection, hyperparameter tuning, and operating-point calibration; n = 25 946) and evaluation (n = 2947) and three "external" centers for calibration (n = 400) and evaluation (n = 16 764). Internal centers contributed developmental data, whereas external centers did not. Deep neural networks predicted the presence of ICH and subtypes (intraparenchymal, intraventricular, subarachnoid, subdural, and/or epidural hemorrhage) and segmentations per case. Two ICH confidence scores are discussed: a calibrated classifier entropy score and a Dempster-Shafer score. Evaluation was completed by using receiver operating characteristic curve analysis and report turnaround time (RTAT) modeling on the evaluation set and on confidence score-defined subsets using bootstrapping.
Results:
The areas under the receiver operating characteristic curve for ICH were 0.97 (0.97, 0.98) and 0.95 (0.94, 0.95) on internal and external center data, respectively. On 80% of the data stratified by calibrated classifier and Dempster-Shafer scores, the system improved the Youden indexes, increasing them from 0.84 to 0.93 (calibrated classifier) and from 0.84 to 0.92 (Dempster-Shafer) for internal centers and increasing them from 0.78 to 0.88 (calibrated classifier) and from 0.78 to 0.89 (Dempster-Shafer) for external centers (P < .001). Models estimated shorter RTAT for AI-prioritized worklists with confidence measures than for AI-prioritized worklists without confidence measures, shortening RTAT by 27% (calibrated classifier) and 27% (Dempster-Shafer) for internal centers and shortening RTAT by 25% (calibrated classifier) and 27% (Dempster-Shafer) for external centers (P < .001).
Conclusion:
AI that provided statistical confidence measures for ICH detection on NCCT scans reliably detected and subtyped hemorrhages, identified high-confidence predictions, and improved worklist prioritization in simulation.Keywords: CT, Head/Neck, Hemorrhage, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2022.
