Performance Evaluation of a Commercial Deep Learning Software for Detecting Intracranial Hemorrhage in a Pediatric
Hadiseh Kavandi1, Kyle Costenbader1, Sandrine Yazbek1
1Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD, USA.
Insights
This study shows an AI tool trained on adults can detect intracranial hemorrhage (ICH) in children with 96% sensitivity. Pediatric-specific features like calcifications caused false positives, indicating a need for tailored AI training.
Area of Science:
- Medical Imaging
- Artificial Intelligence in Medicine
- Pediatric Radiology
Background:
- Timely diagnosis of intracranial hemorrhage (ICH) is critical in pediatric patients.
- Existing artificial intelligence (AI) tools for ICH detection are primarily trained on adult data, creating a research gap in pediatric applications.
- There is a need to evaluate the efficacy of commercially available AI tools in pediatric populations.
Purpose of the Study:
- To evaluate a commercially available AI tool, Aidoc, originally trained on adults, for its performance in detecting intracranial hemorrhage (ICH) in pediatric patients.
- To address the need for accurate and timely ICH diagnosis in children using AI.
- To identify limitations and areas for improvement in AI algorithms for pediatric neuroimaging.
Main Methods:
- A retrospective study included 2502 pediatric patients (aged 6-17) who underwent head CT scans between January 2017 and November 2022.
- Radiological reports and CT images were analyzed by natural language processing (NLP) and image-based AI algorithms, respectively.
- Discrepant cases between AI and NLP were reviewed by three radiologists to establish a reference standard.
Main Results:
- The AI algorithm flagged 292 out of 2502 patients for suspected ICH.
- The AI demonstrated a sensitivity of 96.0% and specificity of 93.7% for ICH detection in pediatric patients.
- Common false positives included choroid plexus calcifications and hyperdense venous sinuses; false negatives were most frequently subdural hemorrhages.
Conclusions:
- The deep learning AI algorithm, trained on adult data, performs with high sensitivity and specificity in detecting pediatric ICH.
- Pediatric-specific findings like calcifications and venous sinuses represent challenges for adult-trained AI, leading to false positives.
- Further development and pediatric-focused training of AI algorithms are necessary to enhance diagnostic accuracy for ICH in the pediatric population.
Abstract:
This study evaluates a commercially available AI tool (Aidoc) for intracranial hemorrhage (ICH) detection-originally trained on adults-in pediatric patients, addressing the critical need for timely diagnosis and current research gaps in pediatric AI applications. This single-center, retrospective study included pediatric patients aged 6-17 who underwent head CT between January 2017 and November 2022. Radiological reports (unaided by AI) and CT images were analyzed by natural language processing (NLP) and image-based algorithms, respectively, to classify ICH presence or absence. Ground truth was assumed for concordant cases. Three radiologists independently reviewed discrepant cases using majority vote. Among 2502 pediatric patients undergoing head CT, the AI algorithm flagged 292 cases as suspected ICH-positive. A total of 174 discordant cases between NLP and AI were independently reviewed to create the reference standard. Results showed 144 true positives, 6 false negatives, 148 false positives, and 2204 true negatives, yielding sensitivity of 96.0% (91.5-98.5%) and specificity of 93.7% (92.6-94.7%). Overall algorithm accuracy was 93.8% (92.8-94.8%). The most frequent false positives were choroid plexus calcifications and hyperdense venous sinuses, while subdural hemorrhages accounted for most false negatives. This deep learning AI algorithm trained on adult data performs well in detecting pediatric ICH, with 96.0% sensitivity and 93.7% specificity. However, common false positives, choroid plexus calcifications and hyperdense venous sinuses, reflect pediatric-specific features, while missed subdural hemorrhages mirror known adult limitations. Results highlight the need for pediatric-focused AI training to improve diagnostic accuracy in this underserved population.
Related Concept Videos
02:04Conservation of Small Populations
01:25What is Population Genetics?
00:57Population Growth
00:30What are Populations and Communities?
01:14Avoidance Learning and Learned Helplessness
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
01:26Pharmacokinetics in Pediatric Patients: Drug Excretion