External Validation of an Artificial Intelligence Device for Intracranial Hemorrhage Detection

Artificial intelligence applications have gained traction in the field of cerebrovascular disease by assisting in the triage, classification, and prognostication of both ischemic and hemorrhagic stroke. The Caire ICH system aims to be the first device to move into the realm of assisted diagnosis for intracranial hemorrhage (ICH) and its subtypes.

A single-center retrospective dataset of 402 head noncontrast CT scans (NCCT) with an intracranial hemorrhage were retrospectively collected from January 2012 to July 2020; an additional 108 NCCT scans with no intracranial hemorrhage findings were also included. The presence of an ICH and its subtype were determined from the International Classification of Diseases-10 code associated with the scan and validated by an expert panel. We used the Caire ICH vR1 to analyze these scans, and we evaluated its performance in terms of accuracy, sensitivity, and specificity.

We found the Caire ICH system to have an accuracy of 98.05% (95% confidence interval [CI]: 96.44%-99.06%), a sensitivity of 97.52% (95% CI: 95.50%-98.81%), and a specificity of 100% (95% CI: 96.67%-100.00%) in the detection of ICH. Experts reviewed the 10 incorrectly classified scans.

The Caire ICH vR1 algorithm was highly accurate, sensitive, and specific in detecting the presence or absence of an ICH and its subtypes in NCCTs. This work suggests that the Caire ICH device has potential to minimize clinical errors in ICH diagnosis that could improve patient outcomes and current workflows as both a point-of-care tool for diagnostics and as a safety net for radiologists.